how to scale to millions of users with aws
TRANSCRIPT
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ken Payne, AWS Solutions Architecture
Jean Meyer, CEO at Once Dating AG
7th July 2016
Scaling the Platform for your Startup
Why are you here?
Building the technology platform for your startup
You want to prepare for success
Learn about design patterns & scalability
A pragmatic approach for startups
Priorities for startups
Racing within a window of opportunity
Small team with no legacy
Focus on solving a problem
Avoid over-engineering & re-engineering
Reduce risk of failure when you go viral
A scalable architecture
Can support growth in users, traffic, data size
Without practical limits
Without a drop in performance
Seamlessly - just by adding more resources
Efficiently - in terms of cost per user
Single host
THE s erver(e.g . Apac he,
MyS Q L)
Elas tic IP
www.example.c om
Amazon Route 53DNS s ervic e
S erver Imag e (AMI)
We need a bigger server
Add larger & faster storage (EBS)
Use the right instance type
Easy to change instance sizes
Not our long term strategy
Will hit an endpoint eventually
No fault toleranceX1 instances offer 1,952 GiB of DDR4
based memory, powered by four Intel®
Xeon® E7 8880 v3 (Haswell)
processors that offer 128 vCPUs
Separating web and DB
More capacity
Scale each tier individually
Tailor instance for each tier
• Instance type
• Storage
Security
• Security groups
• DB in a private VPC subnet
Start with a Relational DB
SQL is versatile & feature-rich
Lots of existing code, tools, knowledge
Clear patterns to scalability (for read-heavy apps)
Reality: eventually you will have a polyglot data layer
• There will be workloads where NoSQL is a better fit
• Combination of both Relational and NoSQL
• Use the right tool for each workload
Key Insight: Relational Databases are Complex
Our experience running Amazon.com taught us that
relational databases can be a pain to manage and operate
with high availability
Poorly managed relational databases are a leading cause
of lost sleep and downtime in the IT world!
Especially for startups with small teams
Relational Databases
MySQL, Aurora, PostgreSQL, Oracle, SQL Server
Fully managed; Zero AdminAmazon
RDS
Aurora
Offload static content
Amazon S3: highly available hosting that scales
• Static files (JavaScript, CSS, images)
• User uploads
S3 URLs – serve directly from S3
Let the web server focus on dynamic content
Amazon CloudFrontWorldwide network of edge locations
Cache on the edge
• Reduce latency
• Reduce load on origin servers
• Static and dynamic content
• Even few seconds caching of popular content can have huge impact
Connection optimizations
• Optimize transfer route
• Reuse connections
• Benefits even non cachable content
C loudFront
CloudFront for static & dynamic content
AmazonRoute 53
EC 2 instanc e(s)
S 3 buc ket
S tatic C ontent
Dynamic C ontent
css/*
js/*
Images/*
Default(*)
C loudFrontDistribution
Database CachingFaster response from RAM
Reduce load on database
Application server
1. If data in cache,
return result
2. If not in cache,
read from DBRDS database
Amazon ElastiC ache
3. And store
in cache
“Lazy Loading”
Amazon ElastiCache: in-memory cache
Simple to Deploy
Managed
• Automatically replaces failed nodes
• Patch management
Elastic
CompatibleElastiCache
High Availability
Availability Zone a
RDS DB ins tanc e
Web s erver
S 3 buc ket fors tatic as s ets
www.example.c om
Amazon Route 53DNS servic e
Amazon C loudFront
Elas tiC ac henode 1
High Availability
Availability Zone a
RDS DB ins tanc e
Availability Zone b
Web s erver
Web s erver
S 3 buc ket fors tatic as s ets
www.example.c om
Amazon Route 53DNS servic e
Amazon C loudFront
Elas tiC ac henode 1
High Availability
Availability Zone a
RDS DB ins tanc e
Availability Zone b
www.example.c om
Amazon Route 53DNS servic e
Elas tic LoadBalanc ing
Web s erver
Web s erver
S 3 buc ket fors tatic as s ets
Amazon C loudFront
Elas tiC ac henode 1
Elastic Load Balancing
Managed Load Balancing Service
Fault tolerant
Health Checks
Distributes traffic across AZs
Elastic – automatically scales its capacity
High Availability
Availability Zone a
RDS DB ins tanc e
Availability Zone b
www.example.c om
Amazon Route 53DNS servic e
Elas tic LoadBalanc ing
Web s erver
Web s erver
S 3 buc ket fors tatic as s ets
Elas tiC ac henode 1
Amazon C loudFront
High Availability
Availability Zone a
RDS DB ins tanc e
Availability Zone b
www.example.c om
Amazon Route 53DNS servic e
Elas tic LoadBalanc ing
Web s erver
Web s erver
RDS DBs tandby
S 3 buc ket fors tatic as s ets
Elas tiC ac henode 1
Amazon C loudFront
Data layer HA
Availability Zone a
RDS DB ins tanc e
Elas tiC ac henode 1
Availability Zone b
S 3 buc ket fors tatic as s ets
www.example.c om
Amazon Route 53DNS servic e
Elas tic LoadBalanc ing
Web s erver
Web s erver
RDS DBs tandby
Data layer HA
Availability Zone a
RDS DB ins tanc e
Elas tiC ac henode 1
Availability Zone b
S 3 buc ket fors tatic as s ets
www.example.c om
Amazon Route 53DNS servic e
Elas tic LoadBalanc ing
Web s erver
Web s erver
RDS DBs tandby
Elas tiC ac henode 2
User Sessions?Problem: Often stored on local disk
(not shared)
Quick-Fix: ELB Session stickiness
Solution: DynamoDB
Elas tic LoadBalanc ing
Web s erver
Web s erver
Logged in Logged out
Amazon DynamoDB
Managed document and key-value store
Simple to launch and scale
• To millions of IOPS
• Both reads and writes
Consistent, fast performance
Durable: perfect for storage of session data
https://github.com/aws/aws-dynamodb-session-tomcat
http://docs.aws.amazon.com/aws-sdk-php/guide/latest/feature-dynamodb-session-handler.html
Replace guesswork with elastic IT
Startups pre-AWS
Demand
Unhappy Customers
Waste $$$
Traditional
Capacity
Capacity
Demand
AWS Cloud
Scaling the web tier
Availability Zone a
RDS DB ins tanc e
Elas tiC ac henode 1
Availability Zone b
S 3 buc ket fors tatic as s ets
www.example.c om
Amazon Route 53DNS servic e
Elas tic LoadBalanc ing
Web s erver
Web s erver
RDS DBs tandby
Elas tiC ac henode 2
Scaling the web tier
Availability Zone a
RDS DB ins tanc e
Elas tiC ac henode 1
Availability Zone b
S 3 buc ket fors tatic as s ets
www.example.c om
Amazon Route 53DNS servic e
Elas tic LoadBalanc ing
Web s erver
Web s erver
RDS DBs tandby
Elas tiC ac henode 2
Web s erver
Web s erver
Scaling the web tier
Availability Zone a
RDS DB ins tanc e
Elas tiC ac henode 1
Availability Zone b
S 3 buc ket fors tatic as s ets
www.example.c om
Amazon Route 53DNS servic e
Elas tic LoadBalanc ing
Web s erver
Web s erver
RDS DBs tandby
Elas tiC ac henode 2
Web s erver
Web s erver
Automatic resizing of compute
clusters based on demand
Feature Details
Control Define minimum and maximum instance pool sizes and when scaling and cool down occurs.
Integrated to Amazon CloudWatch
Use metrics gathered by CloudWatch to drive scaling.
Instance types Run Auto Scaling for on-demand and Spot Instances. Compatible with VPC.
aws autoscaling create-auto-scaling-group
--auto-scaling-group-name MyGroup
--launch-configuration-name MyConfig
--min-size 4
--max-size 200--availability-zones us-west-2c, us-west-2b
Auto Scaling Trigger auto-scaling policy
Amazon
CloudWatch
What does this mean in practice?
Only store transient data on local disk
Needs to persist beyond a single http request?
• Then store it elsewhere
User uploads
User Sessions
Amazon S3
AWS DynamoDB
Application Data
Amazon RDS
Having decomposed into
small, loosely coupled,
stateless building blocks
You can now Scale out with ease
Having done that…
Having decomposed into
small, loosely coupled,
stateless building blocks
We can also Scale back with ease
Having done that…
Take the shortcut
While this architecture is simple you still need to deal
with:
• Configuration details
• Deploying code to multiple instances
• Maintaining multiple environments (Dev, Test, Prod)
• Maintain different versions of the application
Solution: Use AWS Elastic Beanstalk
AWS Elastic Beanstalk (EB)
Easily deploy, monitor, and scale three-tier web
applications and services.
Infrastructure provisioned and managed by EB
You maintain control.
Preconfigured application containers
Easily customizable.
Support for these platforms:
Loose coupling with SQS
Tight coupling
• Place asynchronous tasks into Amazon SQS• SQS – buffer that protects backend systems• Process at own pace• Respond quickly to end users
S Q S
G et Mes s ag e
Bac k End EC 2 Ins tanc e
Put Mes s ag e
Front End EC 2 Ins tanc e
Mobile
Push
NotificationsMobile
AnalyticsCognito
Cognito
Sync
Analytics
KinesisData
PipelineRedShift EMR
Your Applications
AWS Global Infrastructure
Network
VPCDirect
ConnectRoute 53
Storage
EBS S3 Glacier CloudFront
Database
DynamoDBRDS ElastiCache
Deployment & Management
Elastic
BeanstalkOpsWorks
Cloud
Formation
Code
Deploy
Code
Pipeline
Code
Commit
Security & Administration
CloudWatch ConfigCloud
TrailIAM Directory KMS
Application
SQS SWFApp
Stream
Elastic
TranscoderSES
Cloud
Search
SNS
Enterprise Applications
WorkSpaces WorkMail WorkDocs
Compute
EC2 ELBAuto
ScalingLambdaECS
Stay focused as you scale your team
AWSCloud-Based
Infrastructure
YourBusiness
More Time to Focus onYour Business
Configuring Your Cloud Assets
70%
30%70%
On-PremiseInfrastructure
30%
Managing All of the “Undifferentiated Heavy Lifting”
Scaling Relational DBs
Increase RDS instance specs• Larger instance type
• More storage / more PIOPS
Read Replicas (Master – Slave)• Scale out beyond capacity of single DB instance
• Available in Amazon RDS for MySQL, PostgreSQL and Amazon Aurora
• Replication lag
• Writes => master
• Reads with tolerance to stale data => read replica (slave)
• Reads with need for most recent data => master
Scaling the DB
Web s erver
Web s erver
Web s erver
Web s erver
Availability Zone a
RDS DB ins tanc e
Elas tiC ac henode 1
Availability Zone b
S 3 buc ket fors tatic as s ets
www.example.c om
Amazon Route 53DNS servic e
Elas tic LoadBalanc ing
RDS DBs tandby
Elas tiC ac henode 2
Scaling the DB
Web s erver
Web s erver
Web s erver
Web s erver
Availability Zone a
RDS DB ins tanc e
Elas tiC ac henode 1
Availability Zone b
S 3 buc ket fors tatic as s ets
www.example.c om
Amazon Route 53DNS servic e
Elas tic LoadBalanc ing
RDS DBs tandby
Elas tiC ac henode 2
RDS read replic a
Scaling the DB
Web s erver
Web s erver
Web s erver
Web s erver
Availability Zone a
RDS DB ins tanc e
Elas tiC ac henode 1
Availability Zone b
S 3 buc ket fors tatic as s ets
www.example.c om
Amazon Route 53DNS servic e
Elas tic LoadBalanc ing
RDS DBs tandby
Elas tiC ac henode 2
RDS read replic a
RDS read replic a
What if your app is write-heavy?
Challenge: You will eventually hit the write throughput or
storage limit of the master node
Solutions:
Federation (splitting into multiple DBs based on function)
Sharding (splitting one data set up across multiple hosts)
Database federation• Split up databases by
function/purpose
• Harder to do cross-function
queries
• Essentially delaying the need for
something like sharding/NoSQL
until much further down the line
• Won’t help with single huge
functions/tables
Forums DB
Users DB
Products
DB
Sharded horizontal scaling
• More complex at the
application layer
• ORM support can help
• No practical limit on
scalability
• Operation
complexity/sophistication
• Shard by function or key
space
• RDBMS or NoSQL
User ShardID
002345 A
002346 B
002347 C
002348 B
002349 A
Shard C
Shard B
Shard A
NoSQL data stores
Trade query & integrity features of Relational DBs for
• More flexible data model
• Horizontal scalability & predictable performance
DynamoDB
Provisioned read/write performance per table
Massive and Seamless Scale
Distributed system that can scale both reads and writes
• Sharding + Replicas
Automatic partitioning:
• Data set size growth
• Provisioned capacity increases table
Amazon Route 53DNS servic eNo limit
Availability Zone a
RDS DB ins tanc e
Elas tiC ac henode 2
Availability Zone b
S 3 buc ket fors tatic as s ets
www.example.c om
Elas tic LoadBalanc ing
RDS DBstandby
Elas tiC ac henode 3
RDS read replic a
RDS read replic a
DynamoDB
RDS read replic a
Elas tiC ac henode 4
RDS read replic a
ElastiC ac henode 1
C loudS earc hLambdaS ES S Q S
Continuous
scaling
No servers to
manage
Never pay for idle
– no cold servers
(only happy
accountants)
Benefits of AWS Lambda
Amazon API Gateway: Serverless APIs
Internet
Mobile apps
Websites
Services
AWS Lambda
functions
AWS
API Gateway
cache
Endpoints on
Amazon EC2
Any other publicly
accessible endpointAmazon
CloudWatch
Amazon
CloudFrontAmazon
API Gateway
A quick review
Keep it simple and stateless
Make use of managed self-scaling services
Multi-AZ and AutoScale your EC2 infrastructure
Use the right DB for each workload
Cache data at multiple levels
Simplify operations with deployment tools
Next steps?
READ!
aws.amazon.com/documentation
aws.amazon.com/architecture
aws.amazon.com/start-ups
ASK FOR HELP!
forums.aws.amazon.com
aws.amazon.com/support