AWS CloudWatch Metrics

A metric is a time-ordered set of data points. Every AWS service publishes its own automatically, and you can publish your own, which is where both the value and the surprising bills come from.

What identifies a metric

Three things together:

The crucial detail: a metric is defined by its name plus its complete set of dimension values. CPUUtilization with InstanceId=i-001 and CPUUtilization with InstanceId=i-002 are two separate metrics, billed separately if they are custom.

That is the whole cardinality story, and the reason custom metric bills get out of hand.

aws cloudwatch list-metrics --namespace AWS/EC2 \
  --metric-name CPUUtilization --dimensions Name=InstanceId,Value=i-0abc123

Statistics

CloudWatch stores aggregates, not raw events. When you query, you choose how the data points within each period are combined:

Use percentiles for latency, not averages. An average response time of 200ms can hide a p99 of 8 seconds, which is the experience of your slowest and often most valuable requests. Averages conceal exactly the behaviour you need to see.

aws cloudwatch get-metric-statistics \
  --namespace AWS/ApplicationELB --metric-name TargetResponseTime \
  --dimensions Name=LoadBalancer,Value=app/payments/abc123 \
  --start-time 2026-09-14T00:00:00Z --end-time 2026-09-15T00:00:00Z \
  --period 300 --extended-statistics p50 p99

The EC2 metrics that do not exist

EC2 publishes CPU, network and disk I/O automatically. It does not publish memory usage or disk space used, because the hypervisor cannot see inside the guest operating system.

So the two numbers most useful during an incident are absent unless you install the CloudWatch agent:

sudo dnf install -y amazon-cloudwatch-agent
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
  -a fetch-config -m ec2 -c ssm:AmazonCloudWatch-linux -s

Teams routinely discover this mid-incident, having assumed memory was being collected all along. Put the agent in your base AMI or launch template.

Note that agent metrics are custom metrics and are billed per metric per month, so configure it to publish the few you will actually alarm on rather than everything it can collect.

The same gap affects Compute Optimizer, whose right-sizing recommendations are CPU-weighted unless the agent is supplying memory data.

Resolution and retention

Standard resolution is one minute. High resolution is one second, costs considerably more, and is rarely worth it outside genuinely latency-critical systems.

Retention is automatic and lossy over time:

This matters for capacity planning. You cannot go back a year and look at minute-level detail, because it no longer exists. If you need long-term high-granularity history, export to S3 or use Prometheus with its own retention.

Custom metrics, and the cardinality trap

aws cloudwatch put-metric-data \
  --namespace Acme/Payments \
  --metric-name TransactionLatency \
  --dimensions Environment=production,Service=payments \
  --value 142 --unit Milliseconds

Custom metrics are charged per metric per month, where a metric is one unique combination of name and dimension values.

TransactionLatency + {Environment=production, Service=payments}    = 1 metric
TransactionLatency + {Environment=production, Service=checkout}    = 1 metric
TransactionLatency + {Environment=staging,    Service=payments}    = 1 metric

Three metrics. Fine.

Now add user_id as a dimension and you have one metric per user. A hundred thousand users is a hundred thousand billable metrics, and a bill that arrives as a genuine shock.

Dimensions must be low-cardinality. Environment, service, region, instance type. Values from a small fixed set. Anything that is an identifier belongs in logs, where you can query it without a per-value charge.

Embedded Metric Format

A better pattern for application metrics. Write a specially-structured JSON log line and CloudWatch extracts metrics from it automatically:

{
  "_aws": {
    "Timestamp": 1757894400000,
    "CloudWatchMetrics": [{
      "Namespace": "Acme/Payments",
      "Dimensions": [["Service"]],
      "Metrics": [{ "Name": "TransactionLatency", "Unit": "Milliseconds" }]
    }]
  },
  "Service": "payments",
  "TransactionLatency": 142,
  "requestId": "abc-123",
  "userId": "u-9981"
}

Note what happens here: Service is declared as a dimension so it becomes a metric, while requestId and userId stay as log fields. Queryable in Logs Insights but not multiplying your metric count. That separation is exactly what you want, and EMF makes it natural rather than something you have to remember.

It also avoids a synchronous PutMetricData API call on the request path.

Metric math

Compute new series from existing ones without publishing anything:

[
  { "Id": "errors", "MetricStat": { "Metric": { "Namespace": "AWS/ApplicationELB",
      "MetricName": "HTTPCode_Target_5XX_Count", "Dimensions": [{"Name":"LoadBalancer","Value":"app/payments/abc"}]},
      "Period": 300, "Stat": "Sum" }, "ReturnData": false },
  { "Id": "requests", "MetricStat": { "Metric": { "Namespace": "AWS/ApplicationELB",
      "MetricName": "RequestCount", "Dimensions": [{"Name":"LoadBalancer","Value":"app/payments/abc"}]},
      "Period": 300, "Stat": "Sum" }, "ReturnData": false },
  { "Id": "error_rate", "Expression": "errors / requests * 100", "Label": "Error rate %" }
]

An error rate is far more useful to alarm on than an error count, since ten errors means something different at ten requests per second than at ten thousand. Metric math produces it with no extra instrumentation.

ANOMALY_DETECTION_BAND() builds an expected range from historical patterns, which handles workloads with daily or weekly cycles that a static threshold cannot.

Quick reference

---

Key takeaways

• A metric is its name plus its full set of dimension values. Each combination is separate and separately billed.
• Never use a high-cardinality dimension such as a user or request ID, that is what logs are for.
• EC2 publishes no memory or disk usage without the CloudWatch agent.
• Agent metrics are custom metrics and cost per metric per month.
• Use percentiles for latency. An average hides the p99 that users actually feel.
• Resolution degrades with age: minute-level data is gone after 15 days.
• Embedded Metric Format separates dimensions from log fields naturally and avoids a synchronous API call.
• Alarm on rates, not counts, using metric math.

---

Check your understanding

---

Frequently Asked Questions

Why is there no memory metric for my EC2 instance?

Because EC2 only publishes what the hypervisor can see from outside the guest. CPU, network and disk I/O. Memory usage and filesystem utilisation are internal to the operating system and require the CloudWatch agent to be installed and configured on the instance. The agent publishes them as custom metrics, which carry a per-metric monthly charge, so configure it for the few you will actually alarm on. Installing it in your base AMI avoids discovering the gap during an incident.

How are CloudWatch custom metrics priced?

Per metric per month, where a metric is one unique combination of metric name and dimension values. Three services in two environments publishing one metric name is six metrics. The trap is high-cardinality dimensions: adding a user or request ID creates one metric per user or request, which turns a handful of metrics into hundreds of thousands. Keep dimensions to small fixed sets and put identifiers in logs instead.

Should I use average or percentile statistics?

Percentiles for anything latency-related, averages for throughput and utilisation. An average response time is dominated by fast requests and routinely hides a p99 that is an order of magnitude worse, which is the experience of your slowest users. CloudWatch supports p50, p90, p99 and arbitrary percentiles through --extended-statistics, and alarms can be set on them directly.

How long does CloudWatch keep metric data?

Fifteen months, but with decreasing granularity: one-minute data points for 15 days, five-minute for 63 days, one-hour for 455 days, then deletion. The aggregation is automatic and irreversible, so you cannot go back six months and inspect minute-level detail. It no longer exists. For long-term high-resolution history, export to S3 or use a system such as Prometheus with retention you control.

What is the Embedded Metric Format and why use it?

EMF is a structured JSON log format that CloudWatch parses to extract metrics automatically. You write one log line containing both your metric values and any contextual fields, declaring which fields are dimensions. The fields you do not declare remain queryable in Logs Insights without becoming billable metrics, which cleanly separates low-cardinality dimensions from high-cardinality context. It also avoids a synchronous PutMetricData call on the request path.

What is metric math and when do I need it?

Metric math computes new time series from existing metrics without publishing anything new. Most usefully turning counts into rates. Alarming on "more than 10 errors" means something different at ten requests per second than at ten thousand, whereas an error rate expressed as a percentage is meaningful at any volume. It also provides ANOMALY_DETECTION_BAND(), which learns daily and weekly patterns and flags deviations that a static threshold would either miss or fire on constantly.