The Observability Stack is the Phase 7 monitoring layer of the EveryoneCook infrastructure. It provides comprehensive monitoring, alerting, and visualization for all deployed stacks through CloudWatch dashboards, alarms, and SNS notifications.
Deployment Order: This stack MUST be deployed LAST, after all other stacks (DNS, Certificate, Core, Auth, Backend, Frontend) are deployed.
⚠️ Environment Note: This guide focuses on Development (dev) environment deployment. For staging/production deployments, alarm thresholds and monitoring intervals may be different.
CloudWatch Dashboards (4 dashboards):
CloudWatch Alarms (15+ alarms):
Notification System:
┌──────────────────────────────────────────────────────────────────────┐
│ Observability Stack (Phase 7 - Dev Environment) │
│ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ SNS Topic (Alarm Notifications) │ │
│ │ ├─ Topic Name: EveryoneCook-dev-Alarms │ │
│ │ ├─ Email Subscription: team@everyonecook.cloud │ │
│ │ └─ Protocol: Email (requires confirmation) │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ (Alarm Actions) │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ CloudWatch Alarms (15+ alarms) │ │
│ │ │ │
│ │ API Gateway Alarms: │ │
│ │ ├─ 5XX Error Rate > 5% (Critical) │ │
│ │ ├─ 4XX Error Rate > 20% (Warning) │ │
│ │ └─ P99 Latency > 3s (Warning) │ │
│ │ │ │
│ │ Lambda Alarms: │ │
│ │ ├─ Error Rate > 5% (Critical) │ │
│ │ ├─ Throttles > 10 (Critical) │ │
│ │ └─ P99 Duration > 10s (Warning) │ │
│ │ │ │
│ │ DynamoDB Alarms: │ │
│ │ ├─ Read Throttles > 10 (Critical) │ │
│ │ ├─ Write Throttles > 10 (Critical) │ │
│ │ └─ P99 Latency > 100ms (Warning) │ │
│ │ │ │
│ │ S3 Alarms: │ │
│ │ ├─ 4XX Error Rate > 5% (Warning) │ │
│ │ └─ 5XX Errors > 0 (Critical) │ │
│ │ │ │
│ │ SQS Alarms: │ │
│ │ ├─ DLQ Messages > 0 (Critical) │ │
│ │ └─ Message Age > 5 minutes (Warning) │ │
│ │ │ │
│ │ Cost Alarms: │ │
│ │ ├─ Daily Cost > $50 (Warning) │ │
│ │ └─ Daily Cost > $100 (Critical) │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Composite Alarm (System Health) │ │
│ │ ├─ Name: EveryoneCook-dev-SystemHealth │ │
│ │ ├─ Triggers: ANY critical alarm fires │ │
│ │ └─ Action: Send SNS notification │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ CloudWatch Dashboards (4 dashboards) │ │
│ │ │ │
│ │ 1. Core Dashboard (EveryoneCook-dev-Core): │ │
│ │ ├─ DynamoDB: Read/Write capacity, throttles, latency │ │
│ │ ├─ S3: Requests, errors, bytes transferred │ │
│ │ └─ CloudFront: Requests, error rates, bytes downloaded │ │
│ │ │ │
│ │ 2. Auth Dashboard (EveryoneCook-dev-Auth): │ │
│ │ ├─ Cognito: Sign-ups, sign-ins │ │
│ │ └─ Cognito: Failed authentications │ │
│ │ │ │
│ │ 3. Backend Dashboard (EveryoneCook-dev-Backend): │ │
│ │ ├─ API Gateway: Requests, latency (P50/P95/P99) │ │
│ │ ├─ API Gateway: 4XX/5XX errors │ │
│ │ ├─ Lambda: Invocations, duration, errors, throttles │ │
│ │ └─ SQS: Messages sent, visible, oldest age │ │
│ │ │ │
│ │ 4. Overview Dashboard (EveryoneCook-dev-Overview): │ │
│ │ ├─ System Health: Environment info, region │ │
│ │ ├─ Key Metrics: API requests, latency, Lambda stats │ │
│ │ ├─ Error Trends: API 5XX, Lambda errors (last hour) │ │
│ │ ├─ Cost Tracking: Estimated daily cost, 7-day trend │ │
│ │ └─ Alarm Status: Composite alarm widget │ │
│ └─────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────┘
│
│ Monitors
▼
┌──────────────────┴───────────────────┐
▼ ▼ ▼
Core Stack Auth Stack Backend Stack
(DynamoDB, (Cognito) (API Gateway,
S3, CDN) Lambda, SQS)
infrastructure/lib/stacks/
└── observability-stack.ts # Observability Stack implementation (1175 lines)
File: infrastructure/lib/stacks/observability-stack.ts
/**
* Create SNS Topic for CloudWatch Alarms
* Task 7.4.2 - Step 1
*/
private createAlarmTopic(): sns.Topic {
const topic = new sns.Topic(this, 'AlarmTopic', {
topicName: `EveryoneCook-${this.config.environment}-Alarms`,
displayName: 'Everyone Cook CloudWatch Alarms',
});
// Add email subscription for alarm notifications
topic.addSubscription(
new sns_subscriptions.EmailSubscription(this.config.contact.email)
);
return topic;
}
Configuration: Email subscription requires confirmation via AWS SNS.
// API Gateway: High 5XX Error Rate (Critical)
const api5xxAlarm = new cloudwatch.Alarm(this, 'API-5XX-Critical', {
alarmName: `EveryoneCook-${this.config.environment}-API-5XX-Critical`,
alarmDescription: 'API Gateway 5XX error rate > 5% in 5 minutes',
metric: new cloudwatch.Metric({
namespace: 'AWS/ApiGateway',
metricName: '5XXError',
dimensionsMap: { ApiName: apiName },
statistic: 'Sum',
period: cdk.Duration.minutes(5),
}),
threshold: 5,
evaluationPeriods: 2,
comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD,
treatMissingData: cloudwatch.TreatMissingData.NOT_BREACHING,
});
api5xxAlarm.addAlarmAction(alarmAction);
// API Gateway: High Latency (Warning)
const apiLatencyAlarm = new cloudwatch.Alarm(this, 'API-Latency-High', {
alarmName: `EveryoneCook-${this.config.environment}-API-Latency-High`,
alarmDescription: 'API Gateway P99 latency > 3s in 5 minutes',
metric: new cloudwatch.Metric({
namespace: 'AWS/ApiGateway',
metricName: 'Latency',
dimensionsMap: { ApiName: apiName },
statistic: 'p99',
period: cdk.Duration.minutes(5),
}),
threshold: 3000, // 3 seconds in milliseconds
evaluationPeriods: 2,
comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD,
});
// Lambda: High Error Rate (Critical)
const lambdaErrorAlarm = new cloudwatch.Alarm(this, 'Lambda-Error-Rate', {
alarmName: `EveryoneCook-${this.config.environment}-Lambda-Error-Rate`,
alarmDescription: 'Lambda error rate > 5% in 5 minutes',
metric: new cloudwatch.Metric({
namespace: 'AWS/Lambda',
metricName: 'Errors',
statistic: 'Sum',
period: cdk.Duration.minutes(5),
}),
threshold: 5,
evaluationPeriods: 2,
comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD,
});
// Lambda: Throttles (Critical)
const lambdaThrottleAlarm = new cloudwatch.Alarm(this, 'Lambda-Throttle', {
alarmName: `EveryoneCook-${this.config.environment}-Lambda-Throttle`,
alarmDescription: 'Lambda throttles > 10 in 5 minutes',
metric: new cloudwatch.Metric({
namespace: 'AWS/Lambda',
metricName: 'Throttles',
statistic: 'Sum',
period: cdk.Duration.minutes(5),
}),
threshold: 10,
evaluationPeriods: 1,
});
// DynamoDB: Read Throttles (Critical)
const dynamoReadThrottleAlarm = new cloudwatch.Alarm(this, 'DynamoDB-Read-Throttle', {
alarmName: `EveryoneCook-${this.config.environment}-DynamoDB-Read-Throttle`,
alarmDescription: 'DynamoDB read throttles > 10 in 5 minutes',
metric: new cloudwatch.Metric({
namespace: 'AWS/DynamoDB',
metricName: 'ReadThrottleEvents',
dimensionsMap: { TableName: dynamoTableName },
statistic: 'Sum',
period: cdk.Duration.minutes(5),
}),
threshold: 10,
evaluationPeriods: 2,
});
// DynamoDB: High Latency (Warning)
const dynamoLatencyAlarm = new cloudwatch.Alarm(this, 'DynamoDB-Latency-High', {
alarmName: `EveryoneCook-${this.config.environment}-DynamoDB-Latency-High`,
alarmDescription: 'DynamoDB P99 latency > 100ms in 5 minutes',
metric: new cloudwatch.Metric({
namespace: 'AWS/DynamoDB',
metricName: 'SuccessfulRequestLatency',
dimensionsMap: {
TableName: dynamoTableName,
Operation: 'Query',
},
statistic: 'p99',
period: cdk.Duration.minutes(5),
}),
threshold: 100, // 100ms
evaluationPeriods: 3,
});
/**
* Create Composite Alarm for overall system health
* Task 7.4.2 - Step 2
*/
private createCompositeAlarm(alarms: cloudwatch.IAlarm[]): cloudwatch.CompositeAlarm {
// Filter critical alarms only
const criticalAlarms = alarms.filter(
(alarm) => alarm.alarmName.includes('Critical') || alarm.alarmName.includes('Throttle')
);
const compositeAlarm = new cloudwatch.CompositeAlarm(this, 'SystemHealth', {
compositeAlarmName: `EveryoneCook-${this.config.environment}-SystemHealth`,
alarmDescription: 'Overall system health - triggers if any critical alarm fires',
alarmRule: cloudwatch.AlarmRule.anyOf(
...criticalAlarms.map((alarm) =>
cloudwatch.AlarmRule.fromAlarm(alarm, cloudwatch.AlarmState.ALARM)
)
),
});
compositeAlarm.addAlarmAction(new cloudwatch_actions.SnsAction(this.alarmTopic));
return compositeAlarm;
}
/**
* Create Core Dashboard (DynamoDB, S3, CloudFront)
*/
private createCoreDashboard(props: ObservabilityStackProps): cloudwatch.Dashboard {
const dashboard = new cloudwatch.Dashboard(this, 'CoreDashboard', {
dashboardName: `EveryoneCook-${this.config.environment}-Core`,
});
// DynamoDB Metrics
dashboard.addWidgets(
new cloudwatch.GraphWidget({
title: 'DynamoDB - Read/Write Capacity',
left: [
new cloudwatch.Metric({
namespace: 'AWS/DynamoDB',
metricName: 'ConsumedReadCapacityUnits',
dimensionsMap: { TableName: dynamoTableName },
statistic: 'Sum',
period: cdk.Duration.minutes(5),
}),
new cloudwatch.Metric({
namespace: 'AWS/DynamoDB',
metricName: 'ConsumedWriteCapacityUnits',
dimensionsMap: { TableName: dynamoTableName },
statistic: 'Sum',
period: cdk.Duration.minutes(5),
}),
],
width: 12,
}),
new cloudwatch.GraphWidget({
title: 'DynamoDB - Throttles',
left: [
new cloudwatch.Metric({
namespace: 'AWS/DynamoDB',
metricName: 'ReadThrottleEvents',
dimensionsMap: { TableName: dynamoTableName },
statistic: 'Sum',
period: cdk.Duration.minutes(5),
}),
new cloudwatch.Metric({
namespace: 'AWS/DynamoDB',
metricName: 'WriteThrottleEvents',
dimensionsMap: { TableName: dynamoTableName },
statistic: 'Sum',
period: cdk.Duration.minutes(5),
}),
],
width: 12,
})
);
// S3 and CloudFront metrics...
return dashboard;
}
/**
* Create Overview Dashboard (Aggregated view)
*/
private createOverviewDashboard(props: ObservabilityStackProps): cloudwatch.Dashboard {
const dashboard = new cloudwatch.Dashboard(this, 'OverviewDashboard', {
dashboardName: `EveryoneCook-${this.config.environment}-Overview`,
});
// System Health Header
dashboard.addWidgets(
new cloudwatch.TextWidget({
markdown: `# Everyone Cook - System Overview\n\n**Environment:** ${this.config.environment}\n\n**Region:** ${this.region}`,
width: 24,
height: 2,
})
);
// Key Metrics: API, Lambda, DynamoDB, S3
// Cost Tracking
// Alarm Status Widget
return dashboard;
}
Before deploying the Observability Stack, ensure:
All other stacks deployed:
Stack exports available:
aws cloudformation list-exports --region ap-southeast-1
Expected exports:
EveryoneCook-dev-Core-TableNameEveryoneCook-dev-Core-ContentBucketNameEveryoneCook-dev-Core-DistributionIdEveryoneCook-dev-Auth-UserPoolIdEveryoneCook-dev-Backend-ApiNameEmail configured:
infrastructure/config/dev.tsFile: infrastructure/config/dev.ts
export const devConfig: EnvironmentConfig = {
environment: 'dev',
region: 'ap-southeast-1',
// Email for alarm notifications
contact: {
email: 'your-email@example.com', // ⚠️ Update this
phone: '+1234567890',
},
// Monitoring settings (already configured)
monitoring: {
enableDetailedMonitoring: true,
retainLogs: true,
logRetentionDays: 7,
enableXRay: false, // Disabled for dev to save costs
},
};
⚠️ Important: Update the email address to receive alarm notifications.
Navigate to infrastructure directory:
cd D:\Project_AWS\everyonecook\infrastructure
Synthesize the Observability Stack:
npm run synth
Expected Output (1175 lines):
✨ Synthesis time: 3.5s
Resources:
[+] AWS::SNS::Topic AlarmTopic
[+] AWS::SNS::Subscription AlarmTopic/EmailSubscription
[+] AWS::CloudWatch::Alarm API-5XX-Critical
[+] AWS::CloudWatch::Alarm API-4XX-Warning
[+] AWS::CloudWatch::Alarm API-Latency-High
[+] AWS::CloudWatch::Alarm Lambda-Error-Rate
[+] AWS::CloudWatch::Alarm Lambda-Throttle
[+] AWS::CloudWatch::Alarm Lambda-Duration-High
[+] AWS::CloudWatch::Alarm DynamoDB-Read-Throttle
[+] AWS::CloudWatch::Alarm DynamoDB-Write-Throttle
[+] AWS::CloudWatch::Alarm DynamoDB-Latency-High
[+] AWS::CloudWatch::Alarm S3-4XX-Warning
[+] AWS::CloudWatch::Alarm S3-5XX-Critical
[+] AWS::CloudWatch::Alarm SQS-DLQ-Messages
[+] AWS::CloudWatch::Alarm SQS-Queue-Age
[+] AWS::CloudWatch::Alarm Cost-Warning
[+] AWS::CloudWatch::Alarm Cost-Critical
[+] AWS::CloudWatch::CompositeAlarm SystemHealth
[+] AWS::CloudWatch::Dashboard CoreDashboard
[+] AWS::CloudWatch::Dashboard AuthDashboard
[+] AWS::CloudWatch::Dashboard BackendDashboard
[+] AWS::CloudWatch::Dashboard OverviewDashboard
Outputs:
- AlarmTopicArn
- CompositeAlarmName
- CoreDashboardName
- AuthDashboardName
- BackendDashboardName
- OverviewDashboardName
Screenshot: CDK synth output showing all Observability Stack resources
Open the generated CloudFormation template:
code infrastructure/cdk.out/EveryoneCook-dev-Observability.template.json
Screenshot: Generated CloudFormation template showing SNS Topic, 15+ Alarms, Composite Alarm, and 4 Dashboards
Deploy using CDK:
npx cdk deploy EveryoneCook-dev-Observability --require-approval never
Expected Deployment Time: 2-3 minutes
Deployment Output:
EveryoneCook-dev-Observability: deploying...
EveryoneCook-dev-Observability: creating CloudFormation changeset...
EveryoneCook-dev-Observability
Outputs:
EveryoneCook-dev-Observability.AlarmTopicArn = arn:aws:sns:ap-southeast-1:123456789012:EveryoneCook-dev-Alarms
EveryoneCook-dev-Observability.CompositeAlarmName = EveryoneCook-dev-SystemHealth
EveryoneCook-dev-Observability.CoreDashboardName = EveryoneCook-dev-Core
EveryoneCook-dev-Observability.AuthDashboardName = EveryoneCook-dev-Auth
EveryoneCook-dev-Observability.BackendDashboardName = EveryoneCook-dev-Backend
EveryoneCook-dev-Observability.OverviewDashboardName = EveryoneCook-dev-Overview
Stack ARN:
arn:aws:cloudformation:ap-southeast-1:123456789012:stack/EveryoneCook-dev-Observability/...
✨ Deployment time: 2m 15s
Navigate to CloudFormation Console:
AWS Console → CloudFormation → Stacks
Verify Stack:
EveryoneCook-dev-Observability
Screenshot: CloudFormation stack with CREATE_COMPLETE status and 24 resources
Screenshot: CloudFormation Outputs tab showing all 6 outputs
Navigate to CloudWatch Console → Alarms:
AWS Console → CloudWatch → All alarms
Verify Alarms Created (15+ alarms):
| Alarm Name | Type | Metric | Threshold | Status |
|---|---|---|---|---|
EveryoneCook-dev-API-5XX-Critical | Critical | API 5XX Errors | > 5 | OK |
EveryoneCook-dev-API-4XX-Warning | Warning | API 4XX Errors | > 20 | OK |
EveryoneCook-dev-API-Latency-High | Warning | API P99 Latency | > 3000ms | OK |
EveryoneCook-dev-Lambda-Error-Rate | Critical | Lambda Errors | > 5 | OK |
EveryoneCook-dev-Lambda-Throttle | Critical | Lambda Throttles | > 10 | OK |
EveryoneCook-dev-Lambda-Duration-High | Warning | Lambda P99 Duration | > 10000ms | OK |
EveryoneCook-dev-DynamoDB-Read-Throttle | Critical | DynamoDB Read Throttles | > 10 | OK |
EveryoneCook-dev-DynamoDB-Write-Throttle | Critical | DynamoDB Write Throttles | > 10 | OK |
EveryoneCook-dev-DynamoDB-Latency-High | Warning | DynamoDB P99 Latency | > 100ms | OK |
EveryoneCook-dev-S3-4XX-Warning | Warning | S3 4XX Errors | > 5% | OK |
EveryoneCook-dev-S3-5XX-Critical | Critical | S3 5XX Errors | > 0 | OK |
EveryoneCook-dev-SQS-DLQ-Messages | Critical | SQS DLQ Messages | > 0 | OK |
EveryoneCook-dev-SQS-Queue-Age | Warning | SQS Message Age | > 300s | OK |
EveryoneCook-dev-Cost-Warning | Warning | Daily Cost | > $50 | OK |
EveryoneCook-dev-Cost-Critical | Critical | Daily Cost | > $100 | OK |
Verify Composite Alarm:
EveryoneCook-dev-SystemHealth
Screenshot: CloudWatch Alarms console showing all 15+ alarms with OK status
Screenshot: Composite alarm details for system health monitoring
Navigate to CloudWatch Console → Dashboards:
AWS Console → CloudWatch → Dashboards
Verify Dashboards Created (5 dashboards):
EveryoneCook-dev-Core)Open the dashboard and verify widgets:
DynamoDB Widgets:
S3 Widgets:
CloudFront Widgets:
Screenshot: Core Dashboard showing DynamoDB, S3, and CloudFront metrics
EveryoneCook-dev-Auth)Cognito Widgets:
Screenshot: Auth Dashboard showing Cognito authentication metrics
EveryoneCook-dev-Backend)API Gateway Widgets:
Lambda Widgets:
SQS Widgets:
Screenshot: Backend Dashboard showing API Gateway, Lambda, and SQS metrics
EveryoneCook-dev-Overview)System Health Section:
Key Metrics:
Trends:
Cost Tracking:
Alarm Status:
Screenshot: Overview Dashboard with system health, key metrics, error trends, and cost tracking
| Service | Resource | Quantity | Unit Cost | Total |
|---|---|---|---|---|
| CloudWatch Alarms | Standard alarms | 15 | $0.10/alarm | $1.50 |
| CloudWatch Alarms | Composite alarm | 1 | $0.50/alarm | $0.50 |
| CloudWatch Dashboards | Dashboards (>3) | 1 | $3.00/dashboard | $3.00 |
| CloudWatch Metrics | Standard resolution | Included | Free | $0.00 |
| SNS | Email notifications | <1000 | Free tier | $0.00 |
| CloudWatch Logs | 7-day retention | ~5 GB | $0.50/GB | $2.50 |
| Total | $7.50/month |
Free Tier Benefits:
After deploying the Observability Stack:
Confirm Email Subscription: Check inbox and confirm SNS subscription
Review Dashboards: Familiarize yourself with all 4 dashboards
Test Alarms: Trigger a test alarm to verify notifications
Monitor Costs: Check daily cost tracking in Overview Dashboard
⏭️ Deploy Frontend: Continue to 5.10 Deploy to Amplify
⏭️ Test End-to-End: Test complete application flow and monitor metrics
📊 Review Metrics: After 24 hours, review all dashboards for baseline metrics
You have successfully deployed the Observability Stack with:
1 SNS Topic for alarm notifications
15+ CloudWatch Alarms for critical metrics
1 Composite Alarm for overall system health
4 CloudWatch Dashboards for monitoring
Email Notifications configured and confirmed
Key Achievements:
Total Resources: 24 CloudFormation resources
Deployment Time: ~2-3 minutes
Monthly Cost: ~$7.50 (dev environment)
🎉 Congratulations! You have completed all infrastructure stack deployments. Your EveryoneCook platform now has comprehensive monitoring and observability.
Next: 5.10 Deploy to Amplify to deploy the Next.js frontend application.