Custom metrics in AWS CloudWatch allow developers to publish application-specific data points that are not automatically collected by AWS services. This capability is essential for monitoring business logic, application performance, and custom KPIs.
To emit custom metrics from code, developers use…Custom metrics in AWS CloudWatch allow developers to publish application-specific data points that are not automatically collected by AWS services. This capability is essential for monitoring business logic, application performance, and custom KPIs.
To emit custom metrics from code, developers use the AWS SDK's CloudWatch client. The PutMetricData API is the primary method for publishing custom metrics. Each metric requires a namespace (logical container), metric name, value, and optional dimensions for categorization.
In Python using boto3, you would create a CloudWatch client and call put_metric_data() with your metric specifications. For Java, the CloudWatchClient from AWS SDK v2 provides similar functionality through PutMetricDataRequest objects.
Key considerations when emitting custom metrics include:
1. **Batching**: You can send up to 20 metrics per PutMetricData call, reducing API calls and costs.
2. **Dimensions**: These key-value pairs help filter and aggregate metrics. Each metric supports up to 30 dimensions.
3. **Resolution**: Standard resolution stores data at 1-minute granularity, while high resolution captures data at 1-second intervals for an additional cost.
4. **Units**: Specifying units (Seconds, Bytes, Count, etc.) enables proper aggregation and display.
5. **Timestamps**: Metrics can include timestamps up to two weeks in the past or two hours in the future.
For Lambda functions, the Embedded Metric Format (EMF) offers an efficient alternative. By printing structured JSON logs matching the EMF specification, CloudWatch automatically extracts metrics from log data, eliminating separate API calls.
Best practices include implementing retry logic with exponential backoff, using async publishing to avoid blocking application threads, and aggregating data points locally before publishing to minimize costs.
Common troubleshooting issues involve IAM permissions (requiring cloudwatch:PutMetricData), incorrect namespace formatting, and timestamp validation errors. Monitoring the ThrottledRequests metric helps identify rate limiting issues when publishing high volumes of custom metrics.
Emitting Custom Metrics from Code - AWS Developer Associate Guide
Why Custom Metrics Are Important
Custom metrics allow you to monitor application-specific data that AWS CloudWatch does not collect by default. While CloudWatch provides standard metrics like CPU utilization and network traffic, your applications often need to track business-specific measurements such as user logins, transaction counts, error rates, or processing times. Custom metrics bridge this gap, enabling comprehensive observability of your applications.
What Are Custom Metrics?
Custom metrics are user-defined metrics that you publish to Amazon CloudWatch from your application code. They extend CloudWatch's monitoring capabilities beyond the default AWS service metrics. You can define any metric that matters to your application, specify dimensions to segment data, and set up alarms based on these metrics.
How Custom Metrics Work
Custom metrics are published to CloudWatch using the PutMetricData API call. Here's how the process works:
1. Create Metric Data: Define your metric name, namespace, value, unit, timestamp, and optional dimensions
2. Use the AWS SDK: Call the CloudWatch PutMetricData API using the AWS SDK for your programming language (Python/Boto3, Java, Node.js, etc.)
3. Specify a Namespace: Custom metrics must include a namespace (a container for metrics) that does not start with 'AWS/'
4. Add Dimensions: Optionally add dimensions to categorize and filter your metrics (up to 30 dimensions per metric)
5. Set Resolution: Choose standard resolution (1-minute) or high resolution (1-second) for your metrics
Key Components of PutMetricData
- Namespace: Required container for your metrics (e.g., 'MyApplication/Orders') - MetricName: The name of your metric (e.g., 'OrdersProcessed') - Value: The numeric value to record - Unit: The unit of measurement (Count, Seconds, Bytes, Percent, etc.) - Timestamp: When the data point occurred (optional, defaults to current time) - Dimensions: Name-value pairs for filtering (e.g., Environment=Production) - StorageResolution: 1 for high resolution, 60 for standard resolution
Code Example Pattern
The typical pattern involves instantiating a CloudWatch client, creating metric data with your values and dimensions, then calling put_metric_data with your namespace and metric data array. You can batch multiple metrics in a single API call (up to 1000 data points per call).
Costs and Considerations
- Custom metrics incur charges per metric per month - High-resolution metrics cost more than standard resolution - API calls to PutMetricData have associated costs - Data retention: 15 months for all metric data
Exam Tips: Answering Questions on Emitting Custom Metrics from Code
1. Remember the API: The key API for publishing custom metrics is PutMetricData - this is frequently tested
2. Namespace Rules: Custom metric namespaces cannot start with 'AWS/' - this prefix is reserved for AWS services
3. Resolution Options: Know that high-resolution metrics (1-second granularity) use StorageResolution=1, while standard uses 60
4. IAM Permissions: Your code needs cloudwatch:PutMetricData permission to publish metrics
5. Batching: Questions may test knowledge of batching - you can send up to 1000 data points in a single PutMetricData call for efficiency
6. Dimensions Limit: Remember the maximum of 30 dimensions per metric
7. Timestamp Behavior: Data points with timestamps older than 2 weeks or more than 2 hours in the future are not accepted
8. SDK vs CLI: Both AWS SDK (in code) and AWS CLI can publish custom metrics - know when each is appropriate
9. Units Matter: Specifying units allows CloudWatch to perform aggregations correctly and enables meaningful alarm thresholds
10. Embedded Metric Format: For Lambda functions, know that the CloudWatch Embedded Metric Format allows emitting metrics through structured log statements as an alternative approach