Building Python 3 Apps on the Serverless Framework

Take the Python Support in the Serverless Framework for a Spin

Posted by Ryan S. Brown on Wed, Jun 21, 2017
In Mini-Project
Tags: lambda, serverless, python, api gateway

The native language of the Serverless Framework is Javascript, since that’s both the default runtime for Lambda and the language the serverless command-line tool is written in. But since version 0.2.1 Serverless has supported deploying services written in Python 3.6 (or 2.7), .NET core, and Java 8 to Lambda.

In this post we’ll start a new Serverless project and use the Python runtime to make a service that gives out the latest and greatest feline facts. We’ll call it catfacts.

At the time of this writing, version 1.16 is the latest version, so all examples will be based on that.

Bootstrapping

There are a few options for installing the Serverless CLI, but the quickest is to do npm install -g serverless. For more installation options, see the installation docs. There’s a command to create a new project and deploy S3 buckets to handle assets and IAM roles.

$ serverless create
    --template aws-python3
    --name catfacts

This sets up the base structure for the project, and you can run serverless create --help to get a full list of the available project templates. The --name option isn’t required, but if you don’t provide it your service will be named “aws-python3” which isn’t very helpful.

$ serverless deploy --stage dev

This will push your project live with a “hello world” Python function. Wait a few minutes for CloudFormation to build your app and then we can add the custom code.

Configuring the Project

Now we need to fill out the config file with the real information about our project. What functions will there be? What HTTP endpoints must be available? What’s included in the package?

# serverless.yml
service: catfacts

# lock us to a pre-2.0 (not yet released) version of the serverless framework to protect us from breaking changes
frameworkVersion: ">=1.16.0 <2.0.0"

# pick our language and provider, this is automatically filled in by the template
provider:
  name: aws
  runtime: python3.6

package:
  include:
    - common/**
    - show.py
    - catfacts.json
  exclude:
    - requirements.txt
    - serverless.yml
    - README.md
    - LICENSE.txt

# now the good stuff: we can define our functions here
functions:
  show:
    # the "show" part of show.handler is the file,
    # and "handler" indicates the function
    handler: show.handler
    events:
      - http:
          # instead of / you can define any HTTP path you like
          # since we just have one endpoint I used /
          path: /
          method: get

With this outlined, you’ll see that we need to have a show.py file and a handler function within it. We’ll get to that soon, after we write our library code.

Writing Shared Code

In this app we won’t have many functions, but that’s not true for a real application. You’ll have some shared code: everyone needs lib code to handle database connections, type conversions, and so on. Let’s write the shared code that all functions will need. In this case, all our functions need to be able to read facts about cats from our JSON file format. Here’s an example file with two fun feline facts.

[
    "The average cat is 70% fluff",
    "When a cat rubs itself against your leg, it is releasing a pheremone to assert its ownership of you to other cats."
]

The format is a simple JSON list. The full fact list is here for you to copy. Put it in the component directory, facts, so it will be deployed along with any function in the component.

Here’s the contents of common/__init__.py that will take our catfacts.json file and return a parsed version.

import json
import os

here = os.path.dirname(os.path.realpath(__file__))

def all_facts():
    with open(os.path.join(here, '../catfacts.json')) as fact_file:
        facts = json.load(fact_file)
        return facts

In the show.py, we can call this and then pick a fact to send back to the user.

import json
import logging
import random

log = logging.getLogger()
log.setLevel(logging.DEBUG)

# import the shared library, now anything in common/ can be referenced as
# `common.something`
import common


def handler(event, context):
    log.debug("Received event {}".format(json.dumps(event)))
    fact = random.choice(common.all_facts())

    return {
        'statusCode': 200,
        'body': json.dumps({'random_fact': fact})
    }

First Deployment

Congratulations! You have the simplest thing that can possibly work. Now it’s time to send it off to the Internet. Serverless has a handy interactive CLI for deploying code, called serverless deploy.

$ serverless deploy
Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (15 KB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
..............
Serverless: Stack update finished...
Service Information
service: catfacts
stage: dev
region: us-east-1
api keys:
  None
endpoints:
  GET - https://2s75j3z966.execute-api.us-east-1.amazonaws.com/dev/show
functions:
  show: catfacts-dev-show

Now the catfacts-py application is live! You can copy the API-Gateway link into your browser, or use curl to test from the command line. You should see something like this:

# you'll have to replace 2s75.... to match your deployment URL
$ curl https://2s75j3z966.execute-api.us-east-1.amazonaws.com/dev/
{"random_fact": "The average cat is 70% fluff"}

Cool, but what if you want better catfacts, with ASCII art? No problem. In the next section we’ll use an external library to prettify our catfacts.

Prettified Cat Facts

To make cat facts that pretty-print, we’ll have to add a URL parameter to signal whether we want the JSON value to be colorful. So we’ll have to find the query parameters on the event sent to the API Gateway.

desired_color = event['queryStringParameters'].get('color')
# this will either be None, or the string passed as the value in ?color=

API Gateway will pass through the value for “color” in the request URL, as in https://...../?color=red. To handle that in code, you can read the event to get the value of the parameter. Here’s the new handler function that takes ?color and returns the response with the right shell escapes.

For coloration, I’ll use the termcolor library to convert color names into the proper shell escapes. Here’s how to use termcolor inside our function.

import termcolor

def handler(event, context):
    log.debug("Received event {}".format(json.dumps(event)))
    fact = random.choice(common.all_facts())

    desired_color = event.get('queryStringParameters', {}).get('color')

    response = {}
    if desired_color:
        response['random_fact'] = termcolor.colored(fact, desired_color)
    else:
        response['random_fact'] = fact

    return {
        'statusCode': 200,
        'body': json.dumps(response)
    }

To see the whole thing together, get the full file here.

Handling Dependencies

To use the termcolor library (which is providing the escape sequences), we need to make sure the library is in our deployment package. In Node.js it’s traditional to have a node_modules directory alongside your code, while Python typically uses system packages or virtual environments for dependencies. In Lambda, you need to have all the Python libraries you want to use included in the deployment zipfile.

To do this, I like to use a directory called vendored/ and a requirements.txt file. Python packaging is outside the scope of this article, but for now you can just add termcolor as a new line in the requirements.txt file.

Once you’ve done that, install the dependency in your vendored directory by running:

# from the project directory
$ pip install -t vendored/ -r requirements.txt
Collecting termcolor (from -r requirements.txt (line 3))
Installing collected packages; termcolor
Successfully installed termcolor-1.1.0

# after running this, the vendored directory should look like this
$ ls vendored
termcolor-1.1.0.dist-info  termcolor.py  termcolor.pyc

With dependencies taken care of, use serverless deploy again to deploy the endpoint and function.

Trying Out Prettification

To test the coloring, you’ll want to use a terminal. I used a subshell to send the output through echo so my terminal interpreted the color codes correctly, but this depends on how your shell is configured.

$ echo $(curl -s https://2s75j3z966.execute-api.us-east-1.amazonaws.com/dev/\?color\=YOURCOLOR )
Color terminal demo

Wrapping Up

In this post, we’ve covered the basics of using Python in Serverless. Most tutorials that are Node.js focused are still really helpful, even for Python developers, since much of Serverless is language-agnostic like the request/response templating, deployment, and plugins. This giant five part series on using TypeScript from ZeroSharp is top notch, and the testing, deployment, and routing concepts apply to Python as well.

The full code for this app is on Github at ryansb/serverless-cat-facts. To use it, clone the repo and run serverless deploy -s dev to kick off the CloudFormation deploy.

For more advanced info about Python dependencies, check out my post on using scikit-learn on Lambda to see how to handle C libraries and include them in Lambda deploy packages.

Keep up with future posts via RSS. If you have suggestions, questions, or comments feel free to email me, ryan@serverlesscode.com .


Tweet this, send to Hackernews, or post on Reddit