Lambda@Edge CloudFront Debugging

James Beswick, writes on Medium:

but there’s a kink in The Edge process that has caught me out a few times. When your code runs at The Edge, the logging happens in the CloudWatch region of that edge server. The logs might not be where you think they will be.

Lesson: if you can’t find your logs and there are no 502 or 503 errors, The Edge is almost certainly logging in another region. It’s definitely logging somewhere.

Useful info on Lambda@Edge and the capturing of event samples and logging. Be careful when dealing with Lambda@Edge CloudFront CloudWatch logs. They won’t always show up in the region where you expect them to be. Very useful to know as you start to build and debug Lambda@Edge functions.

Here is a simple Lambda@Edge function that you can attach to any of the CloudFront events. It logs the CloudFront event and context passed to your Lambda.

exports.handler = (event, context, callback) => {
  const eventType = event.Records[0].cf.config.eventType;
  const responseEvent = eventType.indexOf('-response') > 0;
  
  console.log('DebugEvent ' + eventType 
    + ' -- ' + JSON.stringify(event)
    + ' -- context -- ' + JSON.stringify(context)
  );
  
  callback(null, (responseEvent ? 
      event.Records[0].cf.response : event.Records[0].cf.request));
};

Samples of the CloudFront context (which seems to be the same for all event types), and event objects for each event type (viewer-request, origin-request, origin-response, viewer-response) are packaged in a Github Gist

As James mentions, and you will find through experience, the Lambda@Edge CloudWatch logs are in the region where the CloudFront endpoint the user hit is located. So you will end up with CloudWatch logs all over the world. And CloudWatch by default sets no expiration on logs, which is troublesome. Below is a command-line script that will find and update log retention where it is unset. See the full gist for the code and the required IAM policy.

Update below log retain shell script converted into a proper lambda function

#!/bin/bash
AWS="/usr/local/bin/aws --profile log-retain-mgr --no-paginate --output json"
JQ="/usr/local/bin/jq --raw-output"

for REGION in `$AWS --region us-east-1 ec2 describe-regions | $JQ '.Regions[].RegionName'`
do
  echo "Region $REGION"
  for GROUP in `$AWS --region "$REGION" logs describe-log-groups | $JQ '.logGroups[] | select (has("retentionInDays") | not).logGroupName'`
  do
    echo "  $REGION $GROUP"
    $AWS --region "$REGION" logs put-retention-policy --log-group-name "$GROUP" --retention-in-days 30
  done
done