Arq Backup Monitor using AWS Lambda, API Gateway, S3
November 13, 2018
Arq Backup is a great personal backup product that lets you backup your Mac to your cloud storage provider of choice . I use Arq Backup for all the users of the many computers we have in the house. It works great, is fairly cheap, and gives you a high level of control over
- what gets backed up (folder selection, exclusion rules)
- when it gets backed up (scheduling, pause during hours)
- security of backups (client side encryption)
One challenge is to make sure backups are running successfully for each user on each computer. If a backup doesn’t happen for 30 days I want to know about it.
I have constructed a two-part solution, Backup Marker and Lambda Monitor
Backup Marker
Write a marker meta-data file to AWS S3 after completion of each backup.
- Create a new bucket to hold marker files
arq-example-monitor
- Create a service role for API Gateway that allows
s3:PutObject
into this bucket - Create an API Gateway service that integrates to S3 to upload the file
- Use service role created above
- Create an API Key, Usage Plan
- Set the API Gateway method to require the API Key
- Optionally set custom domain for the API
- Deploy the API
- Add the API Key to your Mac keychain
- Replace
zzz
with API Key /usr/bin/security add-generic-password -a arq-finish-api-key -s arq-finish-api-key -w zzz -U -T /usr/bin/security
- Replace
- Create shell script
arq-finish.sh
to upload the marker file to S3 via API Gateway- See code below, set the base URL of the API
- Put somewhere like
/Users/Shared
- Give world execute,
chmod 755 /Users/Shared/arq-finish.sh
- Tell Arq to execute the shell script after each backup completes
- Set
Arq -> Preferences -> Destinations -> (Select) -> Edit -> Before and After Backup -> Run shell script after backup completes
to/Users/Shared/arq-finish.sh
- If one user has multiple backup destinations pass an additional argument to the script
/Users/Shared/arq-finish.sh photo
- Set
arq-finish.sh
#!/bin/bash
BASE_URL=https://api.example.com/arq
API_KEY=`/usr/bin/security find-generic-password -a arq-finish-api-key -w`
ARG=${1:-main}
BACKUP_SET="`/bin/hostname -s`.`/usr/bin/whoami`.${ARG}"
FILE=${BACKUP_SET}.txt
URL=${BASE_URL}/${FILE}
/usr/bin/curl --header "x-api-key: $API_KEY" --upload-file - $URL << EOF
`/bin/date`
Arq `/usr/bin/defaults read /Applications/Arq.app/Contents/Info CFBundleVersion`
`/usr/bin/uname -a`
EOF
Lamba Monitor
Periodically run a lambda function to check the age of the marker files and send an alert if too old.
- Create SNS Topic
ArqBackupMonitorAlert
and subscribe your email address - Create a service role for Lambda which contains permission to
s3:ListBucket
ands3:GetObject
onarq-example-monitor
, andsns:Publish
onArqBackupMonitorAlert
- Create ArqBackupMonitor Lambda
- Python 3 runtime
- Code below
- Set variables for at least
ARQ_BUCKET
andARQ_SNS_ARN
- Create CloudWatch trigger to execute lambda every 10 days (set temporarily to every 5 minutes so you can test)
ArqBackupMonitor.py
import boto3
import os
from datetime import datetime, timezone, timedelta
def lambda_handler(event, context):
arq_region = os.environ.get('AWS_REGION', 'us-east-1')
arq_max_age_days = int(os.environ.get('ARQ_MAX_AGE_DAYS', '30'))
arq_alert_subject = os.environ.get('ARQ_SNS_SUBJECT', 'Arq Monitor Alert')
arq_bucket = os.environ.get('ARQ_BUCKET', 'arq-example-monitor')
arq_alert_sns_arn = os.environ.get('ARQ_SNS_ARN',
'arn:aws:sns:us-east-1:12121212:ArqBackupMonitorAlert')
too_old=''
oldest_allowed = datetime.now(timezone.utc) - timedelta(days=arq_max_age_days)
session = boto3.Session(region_name=arq_region)
s3 = session.resource('s3')
bucket = s3.Bucket(arq_bucket)
for object in bucket.objects.all():
if object.key.endswith('.txt'):
if object.last_modified < oldest_allowed:
too_old = too_old + object.key[:-4] + ' '
+ object.last_modified.strftime('%Y-%m-%d') + '\n'
if len(too_old) > 0:
msg = 'Arq backups are too old:\n\n' + too_old
print(msg)
sns = session.resource('sns')
topic = sns.Topic(arq_alert_sns_arn)
topic.publish(Subject=arq_alert_subject, Message=msg)
return 'ArqBackupMonitor.Success'
Summary
- After every successful Arq backup upload marker file via API Gateway to AWS S3
- Every 10 days run a Lamba function that reads the marker files
- If any marker file is older than 30 days send a notification to SNS Topic
- SNS Topic sends notifcation via email
Note: Arq has a new product Arq Cloud Backup - full cloud backup solution including storage, like BackBlaze. I have installed Arq Cloud Backup up on my parents’ computers → it works great , “set it and forget it”. It also is incredibly fast, the Wasabi storage backend is speedy .