Lambda CloudWatch Log Retain Manager
October 31, 2018
As mentioned in Lambda@Edge CloudFront Debugging, it is very common to have services, applications, and worldwide CloudFront Edge Locations (especially Lambda@Edge) creating CloudWatch Log Groups in regions across the world. By default new CloudWatch Log Groups have retention set to Never, which is never what I want.
I created a Lambda function that will search through all AWS regions, check the retention period of each CloudWatch Log Group. Any Log Group that has retention of Never will be changed to retention of 30 Days (configurable). I run this Lambda every 10 days via a scheduled CloudWatch rule. So far so good, keeping CloudWatch logs nice and tidy.
import boto3
import os
def lambda_handler(event, context):
default_region = os.environ.get('AWS_REGION', 'us-east-1')
retain_days = int(os.environ.get('RETAIN_DAYS', '30'))
session = boto3.Session()
client = session.client('ec2', region_name=default_region)
for region_dict in client.describe_regions()['Regions']:
region = region_dict['RegionName']
print('Region:', region)
logs = session.client('logs', region_name=region)
log_groups = logs.describe_log_groups()
for log_group in log_groups['logGroups']:
log_group_name = log_group['logGroupName']
if 'retentionInDays' in log_group:
print(region, log_group_name, log_group['retentionInDays'], 'days')
else:
print(region, log_group_name, retain_days, 'days **PUT**')
response = logs.put_retention_policy(
logGroupName=log_group_name,
retentionInDays=retain_days
)
return 'CloudWatchLogRetention.Success'
Although not used in this function, here is an example of the event object passed to the function when triggered by the CloudWatch scheduled rule
{
"version": "0",
"id": "arar40ba-aaaa-bbbb-cccc-81d33314b7",
"detail-type": "Scheduled Event",
"source": "aws.events",
"account": "1212121212",
"time": "2018-10-30T10:22:22Z",
"region": "us-east-1",
"resources": [
"arn:aws:events:us-east-1:1212121212:rule/logrtnmgr"
],
"detail": {}
}