aws re:invent 2016: monitoring, hold the infrastructure: getting the most from aws lambda (dev205)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Matt Williams - Evangelist @ Datadog

November 29, 2016

Monitoring, Hold the Infrastructure

Getting the Most from AWS Lambda

The Plan

Learn about the basics of AWS Lambda

Add AWS Lambda to your infrastructure

Monitoring 101 (the tl;dr version)

Monitoring AWS Lambda

What is AWS Lambda

The next stage in a long progression...

physical

boxes

virtual

machines

on

physical

boxes

containers lambdaAWS

instances

Important AWS Lambda Concepts

TRIGGER

BASED

Important AWS Lambda Concepts

STATELESS

STATELESS

Important AWS Lambda Concepts

SERVERLESS

AS FAR AS YOU ARE CONCERNED

Important AWS Lambda Concepts

Each event will trigger a new Lambda function

There is a safety limit per region of 100

(contact support for changes)

What is a AWS Lambda Function

• Your executable code

• No restrictions

• Priced based on execution time, quantity, and memory

What can trigger AWS Lambda?

Files Amazon S3

Database Updates Amazon DynamoDB

Streams Amazon Kinesis

Messages Amazon SNS, Amazon SES

User Data & Authentication Amazon Cognito

Infrastructure Changes AWS CloudFormation

Logs, Events, Metrics Amazon CloudWatch

IoT Echo, Amazon API Gateway

LambdaLambda Lambda Lambda

$goad

http://benchling.engineering/crispr-aws-lambda/

Other Examples

Bittorrent TrackerHugely scalable using S3

https://blog.zappa.io/posts/zappa-bittorrent-

tracker

Chat BotsEasy to build

https://claudiajs.com/claudia-bot-builder.html

S3 Data LoadingSimpler than the typical Kinesis process

https://www.trek10.com/blog/serverless-

architectures-s3-data-loading/

Skills for the Amazon EchoMake one today in the hall

Best Practices from AWS

Write stateless code

Local vars

Permissions

Minimize startup

Monitor

Delete old functions

Adding AWS Lambda

to your infrastructure

How to Create a AWS Lambda Function

1.Write the code

2.Setup on AWS

3.Debug

AWS Lambda Demo

Frameworks

node-lambda

Native Gotchas

Lambda runs on an Amazon Linux container

…and your dev box probably isn’t Amazon Linux…

…so native NPM modules are problematic

Native npm

1. Launch an Amazon Linux ec2

2. Use a node version manager

3. Install node 4.3

4. Install the native modules you need

5. Package the modules and dl to your box

6. Deploy

Building the demo with node-lambda

Four files:

• package.json

• .env

• event.json

• index.js

Monitoring 101

the tl;dr version

Collecting data is

cheap

Collecting data is

cheap;

Not having it when you

need it can be

expensive

Monitoring AWS Lambda

What are the AWS Lambda Metrics

aws.lambda.duration, min, max, sum

aws.lambda.errors

aws.lambda.invocations

aws.lambda.throttles

Three Options for Monitoring AWS Lambda

1. Add a custom metric to CloudWatch

2. Add metric directly to monitoring application

3. Add a line to the CloudWatch logs

How Datadog Does It

Add line to CloudWatch logs:

MONITORING|unix_epoch_timestamp|value|type|my.metric.name|#tag1:value,tag2

Monitoring example

Monitoring example

Monitoring example

Monitoring example

Matt Williams

Evangelist - Datadog

mattw@datadoghq.com

@technovangelist

Twitter: @list