Jarbs: Painless Lambda Deployment

Jarbs: Painless Lambda Deployment

AWS Lambda was arguably one of the biggest shifts in modern application development in recent history. It turned raw computing power into on-demand building blocks on which you can run small portions of your application in stateless isolation triggered on an event. Given this brave new world of “server-less architecture,” smoothing over the transition for developers from large-scale application development and deployment to the Lambda world is something we’ve been putting a lot of thought into.

There are a few challenges when first starting with Lambda development:

  1. How should projects be structured?
  2. How do you deploy code?
  3. How do you work with multiple environments?

These, among many other questions, were the reason for building a tool around Amazon’s AWS SDKs that makes it easier to move functions from local development to production endpoints. The resulting tool is called jarbs.
Jarbs is an opinionated tool for building and deploying JavaScript functions into a Lambda environment. It handles the project structure scaffolding, dependency management (via NPM), and deployment. It attempts to abstract the details of Lambda deployment so that you can focus on building and testing your functions locally. The tool is self-documenting using the -h|--h flag.

An Example
jarbs new is where it all begins. This command will stand up a new project containing a shell Lambda awaiting your logic. For this post, we’ll create a simple example project that returns names from an employee database file. We’ll use the following flat file to emulate the database to keep things simple:

[ { "first_name": "Sam", "last_name": "Fischer", "age": 48, "job": "???" }, { "first_name": "Jane", "last_name": "Austine", "age": 62, "job": "author" }, { "first_name": "Monty", "last_name": "Piethon", "age": 129, "job": "comedian" }]

We’ll begin by creating our Lambda project using jarbs:

jarbs new employee-utils

You will see NPM running to install the dependencies of our build system (namely BabelJS for ES6+ usage). Along with this, it builds out the project structure and installs your first shell Lambda which, by default, has the same name as your project (employee-utils).

Jarbs will ask if you would like to submit crash logs to our GitHub repository. Choosing ‘yes’ will require you to enter your GitHub credentials. We do not store these, but rather request an API token so that we can create issues in the jarbs GitHub issue tracker, which lets us more easily catch and fix issues you and others experience while using our tool.

Lastly, jarbs will ask you for a function description, which is simple metadata in the package.json that will help you identify your Lambdas if you have more than one Lambda per project.

Once this process is done, you will want to change your working directory to be the root of your project: cd employee-utils.

If you list the files in this directory, you will see the following structure:

├── lambdas│ └── employee-utils│ └── src│ ├── index.js│ └── package.json| └── node_modules| └── ... many things ...└── package.json

The lambdas dir is where your functions reside. You can see the base index.js file where you will add your custom code. The package.json at that Lambda-level dir controls your function’s dependencies when running in Lambda. For any development dependencies, you will add them to the root project.json. This would include testing libraries or other development/testing related task requirements that you do not want deployed along with your Lambda.

Let’s begin writing our function. Open the index.js file and find the following:

export function handler(event, context) { // Your code here context.done(null, { msg: "Hello from Lambda!" });};

This is where you will enter your code. If you need more information about how Lambda works, as well as what the event and context args are, refer to the AWS Lambda docs.

Your event will be the DB file from above, so to print out the names of the employees, you will add the following (remember, we are able to use ES6 here because of BabelJS):

export function handler(event, context) { let names = []; event.forEach(record => { const name = `${record.first_name} ${record.last_name}` names.push(name); }); // print for example's sake console.log(names.join(', ')); // return names for the receiver of the Lambda output context.done(null, { names: names });};

With this code in hand, you can publish it to AWS and invoke it.

jarbs deploy employee-utils

You will need to specify the IAM role via an ARN either using the --role flag, or when prompted at the CLI. You will probably want to ensure this role has the proper access to CloudWatch logging as well, so that you can retrieve the output of the Lambda. AWS provides a good default role for this if you’ve use the console to deploy Lambdas in the past, usually named lambda_basic_execution, and should have something like the following:

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "arn:aws:logs:*:*:*" } ]}

Now you can invoke this Lambda on AWS. You will want to have the DB file specified above saved as a json file somewhere on your file system to send to AWS (for this example, at the root of the project as db.json).

jarbs invoke employee-utils --file db.json

This will print out your invocation input along with the output from AWS when run. My output looks like the following:

Invoking with:[ { "first_name": "Sam", "last_name": "Fischer", "age": 48, "job": "???" }, { "first_name": "Jane", "last_name": "Austine", "age": 62, "job": "author" }, { "first_name": "Monty", "last_name": "Piethon", "age": 129, "job": "comedian" }]START RequestId: 3094754a-9cfe-11e5-983b-9123502b109b Version: $LATEST2015-12-07T16:18:45.645Z 3094754a-9cfe-11e5-983b-9123502b109b Sam Fischer, Jane Austine, Monty PiethonEND RequestId: 3094754a-9cfe-11e5-983b-9123502b109bREPORT RequestId: 3094754a-9cfe-11e5-983b-9123502b109b Duration: 50.39 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 8 MB

You can see from the second line of the returned output, that it printed the names from my database: Sam Fischer, Jane Austine, Monty Piethon.

You could also create additional functions if you desired, say to list employees by age, or list birthdays. For that, you could simply run jarbs new birthday-listing and it would create a new Lambda function scaffold for you to build out.

You can also cleanup functions you no longer wish to have on Lambda via the jarbs rm method. Once you’re done with this toy example for instance, you can jarbs rm employee-utils and it will delete that function from AWS.

Notes
One of the things you will notice is that jarbs prefixes an environment name to your Lambda. In this example, I did not specify an environment, so dev- was used (i.e. dev-employee-utils). This is our current mechanism for isolating environments for Lambdas. If you want to deploy or invoke a stage or production lambda, you will need to add the -e|--env flag to your commands. For example, jarbs deploy -e prod employee-utils will prefix prod- to the Lambda name in AWS. This will eventually change to use the versioning and aliasing capabilities of Lambda, but for now this fits your needs.

You also have access to the NPM build command if you don’t require BabelJS or desire other build steps before deploying to Lambda. This command is in the scripts definitions of the root-level package.json:

"scripts": { "build:function": "babel --optional runtime"},

You can change the build:function method to whatever you like so long as it builds the contents of the lambdas//dest directory (which doesn’t exist until build time).
There are lots of other things jarbs can do. Both the built-in docs as well as the project’s README provide details.

The Future of Jarbs
Eventually, as we continue to understand our own workflows, we plan on building out Lambda event source management, local invocation, multiple runtime environments (Java, Python) and API Gateway integration as well. We are trying not to build functionality for functionality’s sake but rather to eliminate pain points in our workflows and make jarbs responsible for making developers’ lives easier.