microservices on aws: divide & conquer for agility and scalability

Level 200 (Introductory)

Jim Tran, AWS Enterprise Solutions ArchitectJuly 28, 2016

Architecting Microservices on AWSDivide and conquer for agility and scalability:

ENTERPRISE APPS

DEVELOPMENT & OPERATIONSMOBILE SERVICESAPP SERVICESANALYTICS

DataWarehousing

Hadoop/Spark

Streaming Data Collection

Machine Learning

Elastic Search

Virtual Desktops

Sharing & Collaboration

Corporate Email

Backup

Queuing & Notifications

Workflow

Search

Transcoding

One-click App Deployment

Identity

Single Integrated Console

PushNotifications

DevOps Resource Management

Application Lifecycle Management

Containers

Triggers

Resource Templates

TECHNICAL & BUSINESS SUPPORT

Account Management

Support

Professional Services

Training & Certification

Security & Pricing Reports

Partner Ecosystem

Solutions Architects

MARKETPLACE

Business Apps

Business Intelligence

DatabasesDevOps Tools

NetworkingSecurity Storage

RegionsAvailability Zones

Points of Presence

INFRASTRUCTURE

CORE SERVICES

ComputeVMs, Auto-scaling, & Load Balancing

StorageObject, Blocks, Archival, Import/Export

DatabasesRelational, NoSQL, Caching, Migration

NetworkingVPC, DX, DNS

Access Control

Identity Management

Key Management & Storage

Monitoring & Logs

Assessment and reporting

Resource & Usage Auditing

SECURITY & COMPLIANCE

Configuration Compliance

Web application firewall

HYBRIDARCHITECTURE

Data Backups

Integrated App Deployments

DirectConnect

IdentityFederation

IntegratedResource Management

Integrated Networking

API Gateway

Rules Engine

Device Shadows

Device SDKs

Registry

Device Gateway

Streaming Data Analysis

Business Intelligence

MobileAnalytics

* As of 1 Feb 2016

2011 2013 2015

“The Monolith”

Challenges with monolithic software

Long Build/Test/Release Cycles(who broke the build?)

Operationsis a nightmare(module X is failing, who’s the owner?)

Difficult to scale

New releasestake months

Long time to addnew features

Architecture is hard to maintain and evolve

Lack of innovation

Frustrated customers

Lack of agility

Difficult to scale

Lack of innovation

Lack of agility

Difficult to scale

Lack of innovation

Lack of agility

“20080219BonMorningDSC_0022B” by Sunphol Sorakul . No alterations other than cropping. https://www.flickr.com/photos/83424882@N00/3483881705/Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

Monolith development lifecycle

releasetestbuild

delivery pipeline

app(aka the“monolith”)developers

Photo by Sage Ross. No alterations other than cropping. https://www.flickr.com/photos/ragesoss/2931770125/Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

Too much software coupling

Shared libraries

Too much software coupling

Shared libraries

Shared data

Evolving towards microservices

“IMG_1760” by Robert Couse-Baker. No alterations other than cropping. https://www.flickr.com/photos/29233640@N07/14859431605/Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

“service-oriented

architecture

composed of

loosely coupled

elements

that have

bounded contexts”

Adrian Cockcroft (former Cloud Architect at Netflix,now Technology Fellow at Battery Ventures)

“service-oriented

architecture

composed of

loosely coupled

elements

that have

bounded contexts”

Services communicate with each other over the network

“service-oriented

architecture

composed of

loosely coupled

elements

that have

bounded contexts”

You can update the services independently; updating one service doesn’t require changing any other services.

“service-oriented

architecture

composed of

loosely coupled

elements

that have

bounded contexts”

Self-contained; you can update the code without knowing anything about the internals of other microservices

“Do one thing, and do it well”

“Swiss Army” by by Jim Pennucci. No alterations other than cropping. https://www.flickr.com/photos/pennuja/5363518281/Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

“Tools” by Tony Walmsley: No alterations other than cropping. https://www.flickr.com/photos/twalmsley/6825340663/Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

“Do one thing, and do it well”

Anatomy of a Micro-service

Data Store(eg, RDS, DynamoDB

ElastiCache, ElasticSearch)

Application/Logic(code, libraries, etc)

Public API

POST /restaurantsGET /restaurants

Application/Logic(code, libraries, etc)

Avoid Software Coupling

Driversmicro-services

Paymentsmicro-service Location

micro-services

Orderingmicro-services

Restaurantmicro-service

Ecosystem of micro-services

= 50 million deployments a year

Thousands of teams

× Microservice architecture

× Continuous delivery

× Multiple environments

(5708 per hour, or every 0.63 second)

Gilt: Luxury designer brands at members-only prices

... Sale every day at noon EST

Principle 1

Micro-services only rely on each other’s public API

“Contracts” by NobMouse. No alterations other than cropping.https://www.flickr.com/photos/nobmouse/4052848608/

Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

Micro-service A Micro-service B

public API public API

Principle 1: Micro-services only rely on each other’s public API

Principle 1: Micro-services only rely on each other’s public API(Hide Your Data)

Principle 1: Micro-services only rely on each other’s public API(Evolve API in backward-compatible way…and document!)

storeRestaurant (id, name, cuisine)

Version 1.0.0

public API

Micro-service A

storeRestaurant (id, name, cuisine)storeRestaurant (id, name, arbitrary_metadata)addReview (restaurantId, rating, comments)

Version 1.0.0

Version 1.1.0

public API

Micro-service A

storeRestaurant (id, name, cuisine)storeRestaurant (id, name, arbitrary_metadata)addReview (restaurantId, rating, comments)

storeRestaurant (id, name, arbitrary_metadata)addReview (restaurantId, rating, comments)

Version 1.0.0

Version 1.1.0

Version 2.0.0

public API

Micro-service A

Principle 2

Use the right tool for the job

“Tools #2” by Juan Pablo Olmo. No alterations other than cropping.https://www.flickr.com/photos/juanpol/1562101472/

Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

Principle 2: Use the right tool for the job(Embrace polyglot persistence)

DynamoDB

AmazonElasticsearchService

RDSAurora

Principle 2: Use the right tool for the job(Embrace polyglot programming frameworks)

RDSAurora

Principle 2: Use the right tool for the job(Embrace polyglot programming frameworks)

RDSAurora

Let’s builda micro-service!

Restaurant Micro-service

GET /restaurants

POST /restaurants

Approach #1

EC2EC2 EC2 EC2

Elastic Load Balancer

Approach #2

ContainersUsing ECS

EC2EC2 EC2 EC2

AmazonEC2 ContainerService (ECS)

to managecontainers

• Prototype in less than 2 months

• Deployment time: hours minutes

• Each team can now develop itsrespective applications independently

Coursera13 million users from 190 countries1,000 courses from 119 institutions

Approach #3

API Gateway+ Lambda

DynamoDB

Lambda

DynamoDB

Lambda

API Gateway

AWS Lambda

lets you run code

without managing servers

Upload your code(Java, JavaScript,

Python)

Set up your code to trigger from other AWS

services, webservicecalls, or app activity

Lambda automatically

scales

Python)

Lambda automatically

scales

Python)

Pay for only the compute time

you use(sub-second metering)

AWS API Gateway

is the easiest way to deploy micro-services

Create a unified API frontend for

multiplemicro-services

Authenticate and authorize requests

Handles DDoSprotection and API throttling

…as well as monitoring,

logging, rollbacks,

client SDK generation…

Handles DDoSprotection and API throttling

DynamoDB

Lambdato retrieverestaurants

Lambdato store

restaurants

DynamoDB

API Gateway

POST GET

Lambdato store

restaurants

DynamoDB

API Gateway

POST GET

Web UI

Lambdato store

restaurants

DynamoDB

API Gateway

POST GET

Web UI

Lambdato store

restaurants

Highly Scalable• Inherently scalable

Secure• API Gateway acts as “front door”• Can add authN/authZ; or throttle API if needed• S3 bucket policies• IAM Roles for Lambda invocations

Cost-efficient• Only pay for actual micro-service usage

From a few…

Driversmicro-services

Paymentsmicro-service Location

micro-services

Orderingmicro-services

Restaurantmicro-service

…to an ecosystem

Principle 3

Secure Your Services

“security” by Dave Bleasdale. No alterations other than cropping.https://www.flickr.com/photos/sidelong/3878741556/

Image used with permissions under Creative Commons license 2.0,Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

Principle 3: Secure Your Services

• Defense-in-depth• Network level (e.g. VPC, Security Groups, TLS)• Server/container-level• App-level• IAM policies

• Gateway (“Front door”)

• API Throttling

• Authentication & Authorization• Client-to-service, as well as service-to-service• API Gateway: custom Lambda authorizers• IAM-based Authentication• Token-based auth (JWT tokens, OAuth 2.0)

• Secrets management• S3 bucket policies + KMS + IAM• Open-source tools (e.g. Vault, Keywhiz)

API Gateway

Principle 4

Be a good citizenwithin the ecosystem

“Lamington National Park, rainforest” by Jussarian. No alterations other than cropping.https://www.flickr.com/photos/kerr_at_large/87771074/

Hey Sally, we need to call your micro-service to fetch

restaurants details.

Sure Paul. Which APIs you need to call? Once I know

better your use cases I’ll give you permission to register

your service as a client on our service’s directory entry.

Principle 4: Be a good citizen within the ecosystem

Principle 4: Be a good citizen within the ecosystem(Have clear SLAs)

RestaurantMicro-service

15 TPS100 TPS5 TPS20 TPS

Before we let you call our micro-service we

need to understand your use case, expected load

(TPS) and accepted latency

…and many,many others!

Distributed monitoring and tracing• “Is the service meeting its SLA?”• “Which services were involved in a request?”• “How did downstream dependencies perform?”

Shared metrics• e.g. request time, time to first byte

Distributed tracing• e.g. Zipkin, OpenTracing

User-experience metrics

Principle 4: Be a good citizen within the ecosystem(Distributed monitoring, logging and tracing)

Principle 5

More than justtechnology transformation

“rowing on the river in Bedford” by Matthew Hunt. No alterations other than cropping.https://www.flickr.com/photos/mattphotos/19189529/

“Any organization that designs a system will inevitably produce a design whose structure is a copy of the organization’scommunication structure.”

Melvin E. Conway, 1967

Conway’s Law

Decentralize governance and data management

Image from Martin Fowler’s article on microservices, athttp://martinfowler.com/articles/microservices.html

No alterations other than cropping.Permission to reproduce: http://martinfowler.com/faq.html

Decentralize governance and data management

Silo’d functional teams silo’d application architectures

Cross functional teams self-contained services

Non-pizza image from Martin Fowler’s article on microservices, athttp://martinfowler.com/articles/microservices.html

Cross functional teams self-contained services(“Two-pizza teams” at Amazon)

Full ownership

Full accountability

Aligned incentives

“DevOps”

Non-pizza image from Martin Fowler’s article on microservices, athttp://martinfowler.com/articles/microservices.html

Cross functional teams self-contained services(“Two-pizza teams” at Amazon)

Principle 6

Automate Everything

“Robot” by Robin Zebrowski. No alterations other than cropping.https://www.flickr.com/photos/firepile/438134733/

releasetestbuild

Focused agile teams

2-pizza team delivery pipeline service

releasetestbuild

Focused agile teams

releasetestbuild

Focused agile teams

releasetestbuild

Focused agile teams

releasetestbuild

Focused agile teams

releasetestbuild

Focused agile teams

Principle 6: Automate everything

CodeCommit

CodePipeline

CodeDeploy

EC2 ELBAuto

ScalingLambdaECS

DynamoDBRDS ElastiCache SQS SWF

SES SNS

API GatewayCloudWatch Cloud Trail

KinesisElasticBeanstalk

It’s a journey…

Expect challenges along the way…

• Understanding of business domains

• Coordinating txns across multiple services

• Eventual Consistency

• Service discovery

• Lots of moving parts requires increased coordination

• Complexity of testing / deploying / operating a distributed system

• Cultural transformation

Principles of Microservices

1. Rely only on the public API Hide your data Document your APIs Define a versioning strategy

2. Use the right tool for the job Polygot persistence (data layer) Polyglot frameworks (app layer)

3. Secure your services Defense-in-depth Authentication/authorization

6. Automate everything Adopt DevOps

4. Be a good citizen within the ecosystem Have SLAs Distributed monitoring, logging, tracing

5. More than just technology transformation Embrace organizational change Favor small focused dev teams

Benefits of microservices

Rapid Build/Test/Release Cycles

Clear ownership andaccountability

Easier to scaleeach individual micro-service

New releasestake minutes

Short time to addnew features

Easier to maintain and evolve

Increase innovation

Delighted customers

Increased agility

Easier to maintain and evolve system

Faster innovation

Delighted customers

Increased agility

Easier to maintain and evolve system

Faster innovation

Delighted customers

Increased agility

Additional AWS resources:• Zombie Microservices Workshop:

https://github.com/awslabs/aws-lambda-zombie-workshop• Serverless Webapp - Reference Architecture:

https://github.com/awslabs/lambda-refarch-webapp• Microservices with ECS:

https://aws.amazon.com/blogs/compute/using-amazon-api-gateway-with-microservices-deployed-on-amazon-ecs/

• Serverless Service Discovery:https://aws.amazon.com/blogs/developer/serverless-service-discovery-part-1-get-started/

• ECS Service Discovery:https://aws.amazon.com/blogs/compute/service-discovery-an-amazon-ecs-reference-architecture/

• Microservices without the Servershttps://aws.amazon.com/blogs/compute/microservices-without-the-servers

Popular open-source tools:• Serverless – http://serverless.com• Apex - http://apex.run/

https://aws.amazon.com/devops/

Additional resources

Jim Tran

tranjim@amazon.com

Thank you!

microservices on aws: divide & conquer for agility and scalability

Software

divide and conquer for agility and scalability: an...

the not-so-straigh.orward road from microservices to...

microservices for scalability -...

microservices: aren't microservices just soa?

interoperability and scalability with microservices in...

achieving cloud scalability with microservices and devops...

linux* kernel scalability linux* kernel scalability

microservices & containers - pneuron · microservices &...

building microservices .net - dublin microservices ug...

introduction credited with pioneering controller-less wi...

microservices - nys forum home cb..1.pdf · agenda...

next stage of virtualisation: containers · 2018. 4....

to conquer or not to conquer

of microservices and microservices - robert munteanu

gautam khanna€¦ · microservices 2.0, nosql, event...

pipeline delivery continuous€¦ · microservices...

microservices done right: key ingredients for microservices...

beyond microservices: streams, state and scalability ·...

microservices orchestrations with bpm · 4 the principles...

the technical journey to microservices - microservices ux