Interview: Creators of the DEEP Framework -- Serverless Code

If you want to write web applications in AWS Lambda, there are actually quite a few options out there. There’s the DEEP Framework, Serverless (formerly JAWS), PAWS, and Sparta. There are likely more that I’m omitting – holler on Twitter @ryan_sb if you know of one.

Today, we’ll hear from the creators of the DEEP Framework about its origins, what it’s great at, and how to start using it. The big driver for the Mitoc Group behind going serverless was to reduce ongoing maintenance. What better way to maintain fewer servers by not having them?

Not only is there the DEEP Framework itself, but there’s also a “microapp” store at www.deep.mg where developers can buy and sell small pieces of functionality, like a shopping cart or recommendation engine. It’s still early days for their marketplace, but in the future it could enable developers to build full applications with relatively little code.

The Interview

DEEP is a serverless web framework that abstracts web applications and web services from specific cloud providers. This framework enables developers build cloud-native applications or platforms using microservices architecture in a completely serverless approach. We aim to help enterprise software and enterprise organizations to build web applications or web services that can be used across cloud providers, enabling the customers to choose between Amazon Web Services, or Google Cloud Platform, or Microsoft Azure, or others, without having developers to rewrite the code.

The DEEP Marketplace is a public app store built on top of the DEEP Framework, that empowers customers to choose functionality from listed microservices/microapplications and deploy them together as an web app into their own cloud provider accounts with just few clicks; as well as empowering developers to create and publish their microservices/microapplications and monetize them in similar approach to Apple’s App Store. www.deep.mg is currently in private beta, so it’s still very early to talk about it. But, it’s not a secret and to address your curiosity, we have 16 developers publishing microservices/microapplications and 3 customers using them in production.

DEEP Framework is a collection of JavaScript libraries that can be used either in front-end or back-end, depends on the use case. Every library abstracts certain component of an web application or a cloud provider service. They are published through npmjs.com and documented on docs.deep.mg. The full list of libraries and some details can be found in the README.

DEEP Framework was built originally on AWS, but we want and hope to extend support using other cloud providers like Google Cloud Platform or Microsoft Azure. In AWS context, we are using the following services:

Security - IAM and Cognito
Frontend - S3 and CloudFront
Backend - Lambda, API Gateway and SNS
Database - DynamoDB

Our roadmap includes adding support for services like CloudSearch and Elasticsearch, SQS and Kinesis, VPC and Elasticache.

The team that works on DEEP project is pretty small (less than 10 developers). Except @eistrati, nobody had prior experience with AWS. It’s a long story, and to make it short, core members of the team worked together on other projects that turned out to be 80%+ of our time support and maintenance. So we forced ourselves into a serverless approach and learned on the fly everything else required to get the job done.

At the beginning, CloudWatch was our go-to monitoring tool, but it’s very limited in showing the full picture, especially in realtime. We are using an ELK (Elasticsearch + Logstash + Kibana) stack hacked together to solve our measuring and monitoring needs. Now we’re working to transform that solution into a component that any web application or web platform could use, if it’s built with DEEP Framework.

DEEP Framework, DEEP Marketplace, and DEEP CLI (aka deepify) are components of a bigger Platform-as-a-Service that we call Digital Enterprise End-to-end Platform (aka DEEP).

Production pipeline in context of DEEP Framework + DEEP CLI is focused on developers and makes it easier for them to build, test and deploy serverless applications. For example:

deepify server - simulates the entire stack on localhost and helps developers build and test their microservice(s)/microapplication(s) without any need of cloud provider accounts (e.g. AWS accounts)
deepify deploy - deploys the microservice(s)/microapplication(s) on cloud provider account, into specific environment (e.g. prod, stage, test, dev, etc)
Enjoy the web application!

Production pipeline in context of DEEP Framework + DEEP Marketplace is focused on customers, allowing them to search for functionality in UI (no need for technical expertise) and deploy everything in their own cloud provider accounts as an web application. For example:

Search/select wanted functionality (or compatible functionalities) and add it (or them) to shopping cart
Go to checkout page, provide credit card (pay for licensing the code) and provide cloud provider account credentials (point where the code will be loaded)
Enjoy the web application!

In both cases, DEEP ecosystem manages (and encourages, but doesn’t enforce) environments/versioning/aliases for the users (developers or customers). Where appropriate, we ask for confirmations from users and apply changes to existing stacks. At some point in near future, we’d like to build a process that is very close to blue-green deployments.

Well, this is a tricky question. Since almost everything for us was new, we had lots of questions and misperceptions on certain things, but with help and support from the community and AWS, every task that was hard at the beginning turned up easy at the end. The biggest surprise was to be spotted by AWS Lambda team and invited to speak at AWS re:Invent 2015, the breakout session called “Microservices Architecture for Digital Platforms with AWS Lambda, Amazon CloudFront and Amazon DynamoDB”.

DEEP Framework is open source project, built on top of tons of other open source projects from Github. I’ll try to briefly mention some of them (and apologies in advance if I miss anything):

Yes, we have published couple of examples on how to start building and deploying applications with DEEP Framework + DEEP CLI. There’s a hello world app and a todo app inspired by todomvc.com.

Wrapping Up

Thanks so much to the DEEP team for agreeing to be interviewed. In addition to all that, the team has presented at the AWS Loft and posted 7 Steps to Create Serverless Environments on AWS.

As always, if you have an idea, question, or comment, hit me on Twitter @ryan_sb or email me at ryan@serverlesscode.com.

Note: This post is also on Medium if you’d like to leave comments.

Tweet this, send to Hackernews, or post on Reddit