Amazon CloudWatch

Centralized logging and metrics, on-premises collection, custom metrics and alarms that trigger automated responses, and log analysis.

August 29, 20263 min read4 / 15

Centralized logging and monitoring, AWS's built-in observability tool. Lambda, API Gateway, DynamoDB, and most other managed services ship their logs and metrics here automatically, no separate agent or pipeline to wire up.

Why it matters for a serverless system specifically: a request in a design like the fuel pass architecture touches many services in one flow (CloudFront, API Gateway, EventBridge, SQS, Lambda, DynamoDB). CloudWatch is the shared place to inspect what each of them did for a given request, instead of hunting through separate logs per service. See the serverless mindset for why observability has to be built in from day one rather than bolted on after something breaks.

Not just AWS resources

The CloudWatch agent can be installed on on-premises servers, not only AWS-hosted ones, and ships their logs into the same centralized place. A hybrid setup, part AWS, part on-prem, still ends up with one observability surface instead of two separate ones.

Lambda specifically needs zero setup for this: a plain console.log inside a function body shows up in CloudWatch Logs automatically, no agent, no configuration.

Log groups and the retention gotcha

Logs land in log groups, one per source, named by convention as /aws/<service>/<name> for anything AWS creates automatically (a Lambda function's log group, for instance).

The default retention setting is never expire, and CloudWatch bills for stored log data. Left alone, a busy system's logs accumulate cost indefinitely. Set a short retention window, a week is a reasonable default, and if logs need to be kept longer than that, export them to S3 instead, which is far cheaper storage for data that's rarely read.

Subscription filters vs metric filters

Two different mechanisms, easy to mix up:

  • A subscription filter streams matching log data out, in real time, to a Lambda function, Kinesis, Kinesis Data Firehose, or OpenSearch (the service formerly called Elasticsearch). Useful for reacting to or forwarding logs as they arrive.
  • A metric filter turns a pattern found in log data into a numeric CloudWatch metric, which can then drive an alarm. It doesn't move any data anywhere, it just counts or measures.

The cost trap on subscription filters: a Lambda subscription filter invokes the function once per matching log event, and Lambda bills per invocation. A high-volume, frequently-logging system pointed straight at a Lambda subscription can rack up real cost fast. Kinesis Data Firehose, which batches records before handing them off, is the better fit for high-volume streams.

Metrics, alarms, and automated response

Beyond the metrics AWS services publish automatically, CloudWatch supports custom metrics, anything an application wants to track and report itself, plus the metric-filter-derived ones above. Metrics feed alarms, thresholds that fire when a value crosses a line.

The part worth remembering: an alarm isn't just a notification. It can trigger Lambda or another AWS service directly, turning "something crossed a threshold" into an automated response instead of a page someone has to act on manually.

Log analysis

Beyond raw log storage, CloudWatch supports querying and analyzing the logs it's collected, rather than only being a place logs go to sit. Worth reaching for before standing up a separate log analysis pipeline for straightforward querying needs.

Further Reading and Watching