CloudFormation is a tool for specifying groups of resources in a declarative way. Each resource is actually a small block of JSON that CloudFormation uses to create a real version that is up to the specification provided. In this article, we’ll deploy the EBS snapshot and EBS snapshot cleanup functions with CloudFormation. This’ll change the deploy process from a six-step process into a two-step process.
First, we’ll build the CloudFormation resource for the IAM execution role, then we’ll add the resources for the functions themselves. Finally, we’ll add a schedule.
CloudFormation Resource Brief
You can skip this if you use CloudFormation already.
A CloudFormation template is a collection of a few different “data types”; resources, parameters, and mappings. In this example, we’ll only be using resources. An individual resource has a type, parameters, and name.
"MyS3Bucket": {
"Type": "AWS::S3::Bucket",
"Properties": {
"BucketName": "my.s3.bucket"
}
}
The resource name (not the bucket name) is “MyS3Bucket,” which is also the
key in the Resources object (or dict, or hash, depending on your home
language). The type informs what Properties are available, and tells
CloudFormation what validations are needed.
Finally, the Properties object holds all the information needed to create the
resource. In this case, we have a BucketName parameter that takes a string.
If you make a change to the properties and re-upload the template, some resources must be replaced (others can be updated in-place). CloudFormation creates the new resource before deleting the old one, except in cases where naming constraints would prevent it from doing so. One such case is S3 buckets.
IAM Policy Resources
To run our functions, we need an execution role to grant the functions permission to view EC2 instances and take snapshots. For more on the permissions model, see the Lambda permissions docs. In CloudFormation, this looks like:
"EbsBackupExecutionRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": ["lambda.amazonaws.com"]
},
"Action": ["sts:AssumeRole"]
}
]
},
"Path": "/"
}
},
"EbsBackupExecutionPolicy": {
"DependsOn": [
"EbsBackupExecutionRole"
],
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "MyLambdaToMakeLogsRolePolicy",
"Roles": [
{"Ref": "EbsBackupExecutionRole"}
],
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["logs:*"],
"Resource": ["arn:aws:logs:*:*:*"]
},
{
"Effect": "Allow",
"Action": [
"ec2:Describe*",
"ec2:CreateSnapshot",
"ec2:DeleteSnapshot",
"ec2:CreateTags",
"ec2:ModifySnapshotAttribute",
"ec2:ResetSnapshotAttribute"
],
"Resource": ["*"]
}
]
}
}
}
The above policies should look familiar, as it’s almost exactly like the
specification for IAM roles in the AWS management console. The Role is how
the Lambda functions are invoked, and are linked to a Policy to define the
scope of their capabilities.
CloudFormation will manage changes to this role too! All you need to do is update the policy in the JSON template and the changes will be applied when you update the stack. More on stack updates here
Lambda Function Resources
Lambda functions can be specified as CloudFormation resources (here’s the full docs).
There are two functions we need to specify; one to take the EBS snapshots and one to clean up expired snapshots. They look almost identical since they share the same execution environment (Python), runtime limit (one minute), and role (the one defined above).
Lambda functions can be created based on code stored in zipfiles on S3, and for
your convenience I’ve published the code from the earlier posts in the
examples.serverlesscode.com S3 bucket.
"EbsBackupSnapper": {
"Type": "AWS::Lambda::Function",
"DependsOn": [
"EbsBackupExecutionRole",
"EbsBackupExecutionPolicy"
],
"Properties": {
"Code": {
"S3Bucket": "examples.serverlesscode.com",
"S3Key": "2015-11-ebs-snapshots/ebs-snapper.zip"
},
"Role": {
"Fn::GetAtt": ["EbsBackupExecutionRole", "Arn"]
},
"Timeout": 60,
"Handler": "lambda_function.lambda_handler",
"Runtime": "python2.7",
"MemorySize": 128
}
}
Note the Code property, which maps to the examples.serverlesscode.com bucket
and a zipfile I uploaded (and made public). This JSON has everything necessary
to specify the backup function, with the exception of the scheduled event to
invoke it.
Create the CloudFormation Stack
Now that we understand the template, it’s time to deploy it yourself. Before you do, view the template in your browser to see the contents. It’s the same resources from the samples above, assembled to save you some copy-paste.
Press this button
to spin up a copy.
The new stack will be named “EbsScheduledSnapshots” unless you think of something better. Before creating the stack, make sure to check the box to let it create IAM resources.
Now you’re ready to check out your new functions. Go to the Lambda management console and look for the functions prefixed with the stack name (EbsScheduledSnapshots).
Manual Scheduling
As I mentioned earlier, the CloudFormation resource for AWS Lambda functions doesn’t support scheduled events as a trigger for the Lambda function. To work around this, we have to manually add a schedule from the Management Console.
Go to the AWS Lambda console and look for the two functions with names that
start with “EbsScheduledSnapshots.” They should look like
EbsScheduledSnapshots-EbsBackupSnapper-<some numbers and letters> and
EbsScheduledSnapshots-EbsBackupJanitor-<some numbers and letters>.
For both functions, head to the “Event sources” tab and add a new event source.
Now, set the event to run on a daily interval and name it ebs-daily-backup or
similar.
Recap
In this post, we’ve learned how to deploy a CloudFormation stack to automate managing Lambda functions, and about a limitation of the Lambda API (scheduled events are only available in the console).
Coming up, we’ll learn to use Lambda to create custom resources so you can extend CloudFormation yourself. Thanks for reading! Keep up with future posts via RSS.
As always, if you have an idea, question, comment, or want to say hi hit me on twitter @ryan_sb or email me at ryan@serverlesscode.com.