aws re:invent 2016: serverless computing patterns at expedia (svr306) )

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Kuldeep Chowhan

12/01/2016

SVR306

Serverless Computing

Patterns @ Expedia, Inc.

Kuldeep Chowhan

Principal Engineer @ Expedia, Inc.

@this_is_kuldeep

Expedia, Inc., at a glance

Notes: Expedia, Inc. results shown for 2015. 1Gross bookings is an operating and statistical metric that captures the total dollar value, generally inclusive of taxes and fees, of travel services booked by

our customers; gross bookings excludes HomeAway. 2Monthly visits based on data for Brand Expedia (including Travelocity and Wotif), Hotels.com, Orbitz Worldwide, HomeAway, trivago and Hotwire

combined during 2015.

Sources: Overall travel industry growth rate based on Phocuswright data for 2015 Y/Y.

Largest travel company

$60B gross bookings1

$6.6B revenue growing

>2X faster vs. industry

18K+ employees globally

450M+ monthly site visits2

75+ countries served1.5M+ lodging options

for travelers

What to Expect from the Session

• AWS Lambda Usage at Expedia

• Serverless Apps Creation at Expedia

• Different Serverless App architectures

• Demo

AWS Usage @ Expedia

What is AWS Lambda?

• Run code for virtually any type of application

• No Servers to Manage

• Zero administration

• Continuous Scaling

• Sub second Metering

How does AWS Lambda works?

Primer – AWS Lambda

Creation

Serverless Architectures

@ Expedia

CI/CD Platform utilizing

Serverless Computing

CI/CD System Platform utilizing Serverless Computing

GitHub


GitHub

Webhooks

(Push Events)


GitHub

Webhooks

(Push Events)

Amazon API

Gateway


GitHub

Webhooks

(Push Events)

Amazon API

Gateway

Lambda

Decider/Decorator


GitHub

Webhooks

(Push Events)

Amazon API

Gateway

Amazon

SNSLambda

Decider/Decorator


GitHub

Webhooks

(Push Events)

Amazon API

Gateway

Amazon

SNS

Lambda

Deployment

Config

Changes

Lambda

Decider/Decorator


GitHub

Webhooks

(Push Events)

Amazon API

Gateway

Amazon

SNS

Lambda

Deployment

Config

Changes

Amazon

Elasticsearch Service

Lambda

Decider/Decorator


GitHub

Webhooks

(Push Events)

Amazon API

Gateway

Amazon

SNS

Lambda

Deployment

Config

Changes

Lambda

Developers

pushing to

master

directly

Amazon


Lambda

Decider/Decorator


GitHub

Webhooks

(Push Events)

Amazon API

Gateway

Amazon

SNS

Lambda

Deployment

Config

Changes

Lambda

Developers

pushing to

master

directly

Amazon


Amazon

SES

Lambda

Decider/Decorator


GitHub

Webhooks

(Push Events)

Amazon API

Gateway

Amazon

SNS

Lambda

Deployment

Config

Changes

Lambda

App Details

Changes

Lambda

Developers

pushing to

master

directly

Amazon


Amazon

SES

Lambda

Decider/Decorator


GitHub

Webhooks

(Push Events)

Amazon API

Gateway

Amazon

SNS

Lambda

Deployment

Config

Changes

Lambda

App Details

Changes

Lambda

Developers

pushing to

master

directly

Amazon

DynamoDBAmazon


Amazon

SES

Lambda

Decider/Decorator

Control System for Expedia’s

AWS Infrastructure

Governance

Control System for our AWS Infrastructure Governance

AWS

CloudTrail


AWS

CloudTrailS3 Bucket


AWS

CloudTrailS3 Bucket

Collector

Lambda


AWS

CloudTrailS3 Bucket

Collector

LambdaDynamoDB

Events Table


AWS

CloudTrailS3 Bucket

Collector

LambdaDynamoDB

Events Table

CloudWatch

Events -

Schedule


AWS

CloudTrailS3 Bucket

Collector

LambdaDynamoDB

Events Table

CloudWatch

Events -

Schedule

Dispatcher

Lambda


AWS

CloudTrailS3 Bucket

Collector

LambdaDynamoDB

Events Table

CloudWatch

Events -

Schedule

Dispatcher

Lambda

Amazon

SNS


AWS

CloudTrailS3 Bucket

Collector

LambdaDynamoDB

Events Table

CloudWatch

Events -

Schedule

Dispatcher

Lambda

Amazon

SNSValidator

Lambda


AWS

CloudTrailS3 Bucket

Collector

LambdaDynamoDB

Events Table

CloudWatch

Events -

Schedule

Dispatcher

Lambda

Amazon

SNSValidator

Lambda

Amazon

SNS


AWS

CloudTrailS3 Bucket

Collector

LambdaDynamoDB

Events Table

CloudWatch

Events -

Schedule

Dispatcher

Lambda

Amazon

SNSValidator

Lambda

Destroyer

Lambda

Amazon

SNS


AWS

CloudTrailS3 Bucket

Collector

LambdaDynamoDB

Events Table

CloudWatch

Events -

Schedule

Dispatcher

Lambda

Amazon

SNSValidator

Lambda

Destroyer

Lambda

Amazon

SNSAmazon

SNS


AWS

CloudTrailS3 Bucket

Collector

LambdaDynamoDB

Events Table

CloudWatch

Events -

Schedule

Dispatcher

Lambda

Amazon

SNSValidator

Lambda

Destroyer

LambdaACK

Lambda

Amazon

SNSAmazon

SNS


AWS

CloudTrailS3 Bucket

Collector

LambdaDynamoDB

Events Table

CloudWatch

Events -

Schedule

Dispatcher

Lambda

Amazon

SNSValidator

Lambda

Destroyer

LambdaACK

Lambda

DynamoDB

Sim Table

DynamoDB

History Table

Amazon

SNSAmazon

SNS

Test & Learn Dashboard

Aggregations

Test & Learn Dashboard Aggregations

Amazon

DynamoDB

Stream


Amazon

DynamoDB

Stream

Amazon

Lambda


Amazon

DynamoDB

Stream

Amazon

LambdaAmazon

SQS

Using Lambda to trigger

Typeahead Memory Refresh

Typeahead (ESS – Expedia Suggest Service)

• Typeahead is the first feature that users interact with while

searching for what they are looking for, so providing best

possible suggestions to them is very essential

• ElasticCache Redis + IN-MEMORY data for cache misses

• 25 GB Heap Size per EC2 Instance

• Daily refresh of this data and it takes 15 mins to load the data

The Problem

• Currently each instance that runs ESS in prod has to have

enough memory to support this data refresh process while

still serving requests

• When the occupied heap reaches its maximum limit, the

application:

• often freezes resulting in not able to recommend right

suggestions to the customers

• pauses until garbage collection completes discarding the

large, outdated BLOB from memory

Solution

• Eliminate data refresh process via crontab

• Launch N new instances by modifying the Auto Scaling group

every day during refresh time

• Modify Auto Scaling group to terminate N instances using

OldestFirst EC2 instance termination policy when 2N

instances are InService

• Schedule AWS Lambda to own this responsibility everyday to

scale up and down

Using Lambda to scale up the

Auto Scaling group

Using Lambda to scale up the Auto Scaling group

CloudWatch

Events -

Schedule


CloudWatch

Events -

Schedule

Amazon

Lambda


CloudWatch

Events -

Schedule

Amazon

Lambda

Validate

ASG for

Scale UP


CloudWatch

Events -

Schedule

Amazon

Lambda

Validate

ASG for

Scale UP

Update

Reference Tag

on ASG

Abort & Exit

Failure


CloudWatch

Events -

Schedule

Amazon

Lambda

Validate

ASG for

Scale UP

Update

Reference Tag

on ASG

Scale up ASG

Abort & Exit

Failure

Failure


CloudWatch

Events -

Schedule

Amazon

Lambda

Validate

ASG for

Scale UP

Update

Reference Tag

on ASG

Scale up ASG

Notify & Exit

Abort & Exit

Failure

Failure

Failure

Using Lambda to scale down

the Auto Scaling group

Using Lambda to scale down the Auto Scaling group

CloudWatch

Events -

Schedule


CloudWatch

Events -

Schedule

Amazon

Lambda


CloudWatch

Events -

Schedule

Amazon

Lambda

Validate ASG

for Scale

DOWN


CloudWatch

Events -

Schedule

Amazon

Lambda

Validate ASG

for Scale

DOWN

Scale

DOWN ASG

Abort & Exit

Failure


CloudWatch

Events -

Schedule

Amazon

Lambda

Validate ASG

for Scale

DOWN

Scale

DOWN ASG

Remove the

tag from ASG

Abort & Exit

Failure

Failure


CloudWatch

Events -

Schedule

Amazon

Lambda

Validate ASG

for Scale

DOWN

Scale

DOWN ASG

Remove the

tag from ASG

Notify & Exit

Abort & Exit

Failure

Failure

Failure

Traveler Profile Generation

Using Homegrown Lambda

Benefits of Serverless Computing

• Cost

• Operations

• Scale

• Opportunity

• Time to market

What’s next?

• Improve API Gateway integration in Lambda creation process

• Add support for logging to different sources other than

CloudWatch

• Process hotel images using Lambda

Thank you!

Kuldeep Chowhan

@this_is_kuldeep

Remember to complete

your evaluations!

aws re:invent 2016: serverless computing patterns at expedia (svr306) )

Technology