Using Python in the Serverless Framework

Take the Python Support in the Serverless Framework for a Spin

Posted by Ryan S. Brown on Sun, Feb 21, 2016
In Mini-Project
Tags: lambda, serverless, python, api gateway

The native language of the Serverless Framework is Javascript, since that’s both the default runtime for Lambda and the language the serverless command-line tool is written in. But since version 0.2.1 Serverless has supported deploying services to Lambda’s Python 2.7 environment.

In this post we’ll start a new Serverless project and use the Python runtime to make a service that gives out the latest and greatest feline facts. We’ll call it catfacts.py.

At the time of this writing, version 0.4.2 is the latest version, so all examples will be based on that.

Bootstrapping

There are a few options for installing the Serverless CLI, but the quickest is to do npm install -g serverless. For more installation options, see the installation docs. There’s a command to create a new project and deploy S3 buckets to handle assets and IAM roles.

$ serverless project create \
    --region us-east-1 \
    --name catfacts \
    --domain catfacts.serverlesscode.com

This sets up the base structure for the project, you’ll want to specify a different value for the --domain option to avoid conflicting with my project bucket. Wait a few minutes for CloudFormation to build your resources and then we can create the component where our API will live.

$ serverless component create -r python2.7
Serverless: Enter a name for your new component:  (nodejscomponent) facts
Serverless: Installing default python dependencies with pip...

$ serverless function create facts/show

Writing Common Code

Now that we have a function, let’s write the shared code that all functions will need. In this case, all our functions need to be able to read facts about cats from our JSON file format. Here’s an example file with two fun feline facts.

[
    "The average cat is 70% fluff",
    "When a cat rubs itself against your leg, it is releasing a pheremone to assert its ownership of you to other cats."
]

The format is a simple JSON list. The full fact list is here for you to copy. Put it in the component directory, facts, so it will be deployed along with any function in the component.

Here’s the contents of facts/lib/__init__.py that will take our catfacts.json file and return a parsed version.

import json
import os

here = os.path.dirname(os.path.realpath(__file__))

def all_facts():
    with open(os.path.join(here, '../catfacts.json')) as fact_file:
        facts = json.load(fact_file)
        return facts

In the facts/show/handler.py, we can call this and then pick a fact to send back to the user.

import json
import logging
import random

log = logging.getLogger()
log.setLevel(logging.DEBUG)

# this adds the component-level `lib` directory to the Python import path
import sys, os
# get this file's directory independent of where it's run from
here = os.path.dirname(os.path.realpath(__file__))
sys.path.append(os.path.join(here, "../"))
sys.path.append(os.path.join(here, "../vendored"))

# import the shared library, now anything in component/lib/__init__.py can be
# referenced as `lib.something`
import lib


def handler(event, context):
    log.debug("Received event {}".format(json.dumps(event)))
    facts = lib.all_facts()

    return {
        "random_fact": random.choice(facts)
    }

First Deployment

Congratulations! You have the simplest thing that can possibly work. Now it’s time to send it off to the Internet. Serverless has a handy interactive CLI for deploying code, called serverless dash. The hotkey that’s important here is Ctrl-A to select all endpoints and functions, and then use the arrow keys to move down to the Deploy option.


$ serverless dash deploy
---- some output excluded ----
Serverless: Select the assets you wish to deploy:
    facts/show
      function - facts/show
      endpoint - facts/show@show~GET
    - - - - -
  > Deploy
    Cancel

Serverless: Deploying functions in "dev" to the following regions: us-east-1
Serverless: Deploying endpoints in "dev" to the following regions: us-east-1
Serverless: Successfully deployed endpoints in "dev" to the following regions:
Serverless: us-east-1 ------------------------
Serverless: GET - show - https://2s75j3z966.execute-api.us-east-1.amazonaws.com/dev/show

Now the catfacts-py application is live! You can copy the API-Gateway link into your browser, or use curl to test from the command line. You should see something like this:

$ curl https://2s75j3z966.execute-api.us-east-1.amazonaws.com/dev/show
{"random_fact": "The average cat is 70% fluff"}

Cool, but what if you want better catfacts, with ASCII art? No problem. In the next section we’ll use an external library to prettify our catfacts.

Prettified Cat Facts

To make cat facts that pretty-print, we’ll have to add a URL parameter to signal whether we want the JSON value to be colorful. In your s-function.json change the requestTemplates key to:

"requestTemplates": {
    "application/json": {
        "color": "$input.params('color')"
    }
},

This will pass through the value for “color” in the request URL, as in http://...../show?color=red. To handle that in code, you can read the event to get the value of the parameter. Here’s the new handler function that takes ?color and returns the response with the right shell escapes.

For coloration, I’ll use the termcolor library to convert color names into the proper shell escapes. Here’s how to use termcolor inside our function.

import termcolor


def handler(event, context):
    log.debug("Received event {}".format(json.dumps(event)))
    fact = random.choice(lib.all_facts())

    if event.get('color'):
        return {
            'random_fact': termcolor.colored(fact, event.get('color', 'red'))
        }

    return {
        "random_fact": fact
    }

Handling Dependencies

To use the termcolor library (which is providing the escape sequences), we need to make sure the library is in our deployment package. In Node.js it’s traditional to have a node_modules directory alongside your code, while Python typically uses system packages or virtual environments for dependencies. In Lambda, you need to have all the Python libraries you want to use included in the deployment zipfile.

To do this, Serverless recommends using the created-by-default vendored/ directory and a component-level requirements.txt file. Python packaging is outside the scope of this article, but for now you can just add termcolor as a new line in the requirements.txt file.

Once you’ve done that, install the dependency in your vendored directory by running:

# from the module directory, facts/
$ pip install -t vendored/ -r requirements.txt
Collecting termcolor (from -r requirements.txt (line 3))
Installing collected packages; termcolor
Successfully installed termcolor-1.1.0

# after running this, the vendored directory should look like this
$ ls vendored
termcolor-1.1.0.dist-info  __init__.py  termcolor.py  termcolor.pyc

With dependencies taken care of, use serverless dash again to deploy the endpoint and function.

Trying Out Prettification

To test the coloring, you’ll want to use a terminal. I used a subshell to send the output through echo so my terminal interpreted the color codes correctly, but this depends on how your shell is configured.

$ echo $(curl -s https://2s75j3z966.execute-api.us-east-1.amazonaws.com/dev/show\?color\=YOURCOLOR )
Color terminal demo

Wrapping Up

In this post, we’ve covered the basics of using Python in Serverless. Most tutorials that are Node.js focused are still really helpful, even for Python developers, since much of Serverless is language-agnostic like the request/response templating, deployment, and plugins. This giant five part series on using TypeScript from ZeroSharp is top notch, and the testing, deployment, and routing concepts apply to Python as well.

The full code for this app is on Github at ryansb/serverless-cat-facts. To use it, clone the repo and run serverless project init to kick off the CloudFormation deployment, then use serverless dash deploy to deploy your function and API endpoints.

For more advanced info about Python dependencies, check out my post on using scikit-learn on Lambda to see how to handle C libraries and include them in Lambda deploy packages.

Keep up with future posts via RSS. If you have suggestions, questions, or comments feel free to email me, ryan@serverlesscode.com .


Tweet this, send to Hackernews, or post on Reddit