CloudWatch Alarms and EventBridge

Alarms evaluate metrics over windows

An alarm has OK, ALARM, and INSUFFICIENT_DATA states. Evaluation periods and datapoints-to-alarm determine how many breaching periods are required (for example 3 out of 5); missing-data behavior should match the metric.

flowchart LR
  M["Metric<br/>CPUUtilization"] --> A["Alarm<br/>> 80% for 3 of 5 periods"]
  A --> SNS["SNS topic<br/>email, Slack, pager"]
  A --> ASG["Auto Scaling<br/>add instances"]
  A --> EC2["EC2 action<br/>recover, reboot"]
  EV["AWS event<br/>instance stopped"] --> EB["EventBridge rule<br/>pattern match"]
  EB --> L["Lambda, SQS, SNS<br/>targets"]
  %% step M: An alarm watches one metric, or a metric math expression.
  %% step A: It changes state only when enough datapoints breach the threshold, which avoids alerting on one spike.
  %% step SNS: In ALARM it can notify a topic; subscribers get an email or a chat message.
  %% step ASG: The same alarm can drive a scaling policy.
  %% step EC2: Or an EC2 action such as recovering an instance onto new hardware.
  %% step EV: Separately, AWS services emit events whenever something happens.
  %% step EB: An EventBridge rule matches events by pattern, or runs on a schedule.
  %% step L: Matching events are delivered as JSON to targets such as Lambda or SQS.

Alarm settings

An SNS email subscription must be confirmed before receiving alarm notifications. INSUFFICIENT_DATA commonly means too few datapoints or a wrong dimension, not necessarily a service failure. For metrics that report only when something happens (errors), treat missing data as notBreaching.

Composite alarms combine other alarms with AND/OR, reducing noise: page only when high latency AND high error rate occur together. Anomaly detection alarms learn a metric's normal band instead of a fixed threshold.

EC2 status alarms

An EC2 system status check covers underlying AWS infrastructure; an instance status check covers the guest OS or networking. A recover action on StatusCheckFailed_System moves an instance to healthy hardware in supported cases while preserving instance ID, private IPs, Elastic IPs and EBS volumes. A reboot action suits instance-check failures.

EventBridge

CloudWatch Events is now Amazon EventBridge. A rule matches an event pattern or schedule and sends event JSON to a target:

{"source": ["aws.ec2"], "detail-type": ["EC2 Instance State-change Notification"],
 "detail": {"state": ["stopped", "terminated"]}}

Targets include Lambda, SQS, SNS, Step Functions, ECS tasks and other event buses. EventBridge Scheduler runs one-time or recurring jobs (cron or rate) at scale. Alarms react to metric values; EventBridge reacts to things that happened, such as an instance stopping, a CloudTrail API call, or a GuardDuty finding.

Traps: forgetting to confirm the SNS email; alarming on a single datapoint and getting paged for blips; reading INSUFFICIENT_DATA as an outage.