Deploy Lambda Functions With CloudFormation

Use CloudFormation To Deploy Lambda Functions Easily

Posted by Ryan S. Brown on Sat, Nov 28, 2015
In General
Tags: cloudformation, lambda

CloudFormation is a tool for specifying groups of resources in a declarative way. Each resource is actually a small block of JSON that CloudFormation uses to create a real version that is up to the specification provided. In this article, we’ll deploy the EBS snapshot and EBS snapshot cleanup functions with CloudFormation. This’ll change the deploy process from a six-step process into a two-step process.

First, we’ll build the CloudFormation resource for the IAM execution role, then we’ll add the resources for the functions themselves. Finally, we’ll add a schedule.

CloudFormation Resource Brief

You can skip this if you use CloudFormation already.

A CloudFormation template is a collection of a few different “data types”; resources, parameters, and mappings. In this example, we’ll only be using resources. An individual resource has a type, parameters, and name.

"MyS3Bucket": {
    "Type": "AWS::S3::Bucket",
    "Properties": {
        "BucketName": "my.s3.bucket"
    }
}

The resource name (not the bucket name) is “MyS3Bucket,” which is also the key in the Resources object (or dict, or hash, depending on your home language). The type informs what Properties are available, and tells CloudFormation what validations are needed.

Finally, the Properties object holds all the information needed to create the resource. In this case, we have a BucketName parameter that takes a string.


Fun sidebar: If you don’t want to write CloudFormation in JSON, or want to improve your component reuse, check out SparkleFormation. It’s got a great DSL (Domain-Specific Language) for building templates, and a nice command-line tool for deploying your stacks.


If you make a change to the properties and re-upload the template, some resources must be replaced (others can be updated in-place). CloudFormation creates the new resource before deleting the old one, except in cases where naming constraints would prevent it from doing so. One such case is S3 buckets.

IAM Policy Resources

To run our functions, we need an execution role to grant the functions permission to view EC2 instances and take snapshots. For more on the permissions model, see the Lambda permissions docs. In CloudFormation, this looks like:

"EbsBackupExecutionRole": {
    "Type": "AWS::IAM::Role",
    "Properties": {
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Principal": {
                        "Service": ["lambda.amazonaws.com"]
                    },
                    "Action": ["sts:AssumeRole"]
                }
            ]
        },
        "Path": "/"
    }
},
"EbsBackupExecutionPolicy": {
    "DependsOn": [
        "EbsBackupExecutionRole"
    ],
    "Type": "AWS::IAM::Policy",
    "Properties": {
        "PolicyName": "MyLambdaToMakeLogsRolePolicy",
        "Roles": [
            {"Ref": "EbsBackupExecutionRole"}
        ],
        "PolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Action": ["logs:*"],
                    "Resource": ["arn:aws:logs:*:*:*"]
                },
                {
                    "Effect": "Allow",
                    "Action": [
                        "ec2:Describe*",
                        "ec2:CreateSnapshot",
                        "ec2:DeleteSnapshot",
                        "ec2:CreateTags",
                        "ec2:ModifySnapshotAttribute",
                        "ec2:ResetSnapshotAttribute"

                    ],
                    "Resource": ["*"]
                }
            ]
        }
    }
}

The above policies should look familiar, as it’s almost exactly like the specification for IAM roles in the AWS management console. The Role is how the Lambda functions are invoked, and are linked to a Policy to define the scope of their capabilities.

CloudFormation will manage changes to this role too! All you need to do is update the policy in the JSON template and the changes will be applied when you update the stack. More on stack updates here

Lambda Function Resources

Lambda functions can be specified as CloudFormation resources (here’s the full docs).

There are two functions we need to specify; one to take the EBS snapshots and one to clean up expired snapshots. They look almost identical since they share the same execution environment (Python), runtime limit (one minute), and role (the one defined above).

Lambda functions can be created based on code stored in zipfiles on S3, and for your convenience I’ve published the code from the earlier posts in the demos.serverlesscode.com S3 bucket.

"EbsBackupSnapper": {
    "Type": "AWS::Lambda::Function",
    "DependsOn": [
        "EbsBackupExecutionRole",
        "EbsBackupExecutionPolicy"
    ],
    "Properties": {
        "Code": {
            "S3Bucket": "demos.serverlesscode.com",
            "S3Key": "2015-11-ebs-snapshots/ebs-snapper.zip"
        },
        "Role": {
            "Fn::GetAtt": ["EbsBackupExecutionRole", "Arn"]
        },
        "Timeout": 60,
        "Handler": "lambda_function.lambda_handler",
        "Runtime": "python2.7",
        "MemorySize": 128
    }
}

Note the Code property, which maps to the demos.serverlesscode.com bucket and a zipfile I uploaded (and made public). This JSON has everything necessary to specify the backup function, with the exception of the scheduled event to invoke it.


Warning: Scheduling Lambda functions isn’t supported (as of November, 2015) via the API. To actually use these functions you need to manually add a scheduled event source.


Create the CloudFormation Stack

Now that we understand the template, it’s time to deploy it yourself. Before you do, view the template in your browser to see the contents. It’s the same resources from the samples above, assembled to save you some copy-paste.

Press this button Launch stack EbsScheduledSnapshots to spin up a copy.

The new stack will be named “EbsScheduledSnapshots” unless you think of something better. Before creating the stack, make sure to check the box to let it create IAM resources.

Allow IAM management capability for Lambda roles

Now you’re ready to check out your new functions. Go to the Lambda management console and look for the functions prefixed with the stack name (EbsScheduledSnapshots).

Manual Scheduling

As I mentioned earlier, the CloudFormation resource for AWS Lambda functions doesn’t support scheduled events as a trigger for the Lambda function. To work around this, we have to manually add a schedule from the Management Console.

Go to the AWS Lambda console and look for the two functions with names that start with “EbsScheduledSnapshots.” They should look like EbsScheduledSnapshots-EbsBackupSnapper-<some numbers and letters> and EbsScheduledSnapshots-EbsBackupJanitor-<some numbers and letters>.

For both functions, head to the “Event sources” tab and add a new event source.

Add event-source button in event tab

Now, set the event to run on a daily interval and name it ebs-daily-backup or similar.

Configure daily event

Recap

In this post, we’ve learned how to deploy a CloudFormation stack to automate managing Lambda functions, and about a limitation of the Lambda API (scheduled events are only available in the console).

Coming up, we’ll learn to use Lambda to create custom resources so you can extend CloudFormation yourself. Thanks for reading! Keep up with future posts via RSS or subscribe to the mailing list.

As always, if you have an idea, question, comment, or want to say hi hit me on twitter @ryan_sb or email me at ryan@serverlesscode.com.


Tweet this, send to Hackernews, or post on Reddit