aws deck templateaws-de-media.s3.amazonaws.com/images/aws summit berlin... · 2015-06-24 · more...

Berlin

Running Lean and Mean

Designing Cost-Effective Architectures on AWS

Constantin Gonzalez

Principal Solutions Architect, AWS Germany

What you’ll get out of this session

• A lower AWS bill

• A more scalable, robust, dynamic architecture

• More time to innovate

• Real-world customer examples

• Easy to implement

Cloud computing benefits• Pay as you go, no up-front investments

• Low ongoing cost

• Flexible capacity

• Speed, agility, and innovation

• Focus on your business

• Go global in minutes

Cloud computing benefits• Pay as you go, no up-front investments

• Low ongoing cost

• Flexible capacity

• Speed, agility, and innovation

• Focus on your business

• Go global in minutes

Optimizing for Cost

Strategy 1: Do nothing

AWS pricing philosophyEcosystem

Global Footprint

New Features

New Services

More AWS

Usage

More

Infrastructure

Lower

Infrastructure

Costs

Reduced

Prices

More

CustomersInfrastructure

Innovation

48 price

reductions

since 2006Economies

of Scale

Strategy 2:

Do almost nothing

AWS Trusted Advisoraws.amazon.com/premiumsupport/trustedadvisor/

Free with Business or Enterprise Support

Strategy 3:

Optimize Architecture

The basic process

Architecting for low cost

1: Turn off unused instances

1: Turn off unused instances

• Developer, test, training instances

• Use simple instance start and stop

• Or tear down and build up all together

• Instances are disposable

• Automate, automate, automate:– AWS CloudFormation

– Weekend/off-hours scripts

– Use tags

Customer dev/test example

Monday Friday End of Vacation Season35% saved

2: Use Auto Scaling

Auto Scaling

AWS CloudFormation: Launch Configuration

"LaunchConfig": {

"Type" : "AWS::AutoScaling::LaunchConfiguration",

"Metadata" : {

"AWS::CloudFormation::Init" : {

"config" : {

… packages, sources, files, services …

}

}

},

"Properties": {

"ImageId" : "ami-149f7863",

"InstanceType" : "m1.small",

"SecurityGroups" : [ {"Ref" : "WebServerSecurityGroup"} ],

"KeyName" : "MySSHKey",

"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [

"#!/bin/bash -v\n",

… your user data script …

]]}}

}

}

AWS CloudFormation: Auto Scaling group

"WebServerGroup" : {

"Type" : "AWS::AutoScaling::AutoScalingGroup",

"Properties" : {

"AvailabilityZones" : [

"us-east-1a","us-east-1b","us-east-1c",

],

"LaunchConfigurationName" : { "Ref" : "LaunchConfig" },

"MinSize" : “3",

"MaxSize" : “6",

"DesiredCapacity" : “3",

"LoadBalancerNames" : [ { "Ref" : "ElasticLoadBalancer" } ]

}

}

Align your resources with demand

3: Use Reserved Instances

3: Use Reserved Instances

1y RI

Break even

3y RI

Break even

Reserved Instances are flexible

• Can be moved between AZs

• Can be moved between EC2-

Classic and EC2-VPC platforms

• Size can be modified within the

same instance family

Track Savings over Time

Who am I• Markus Ostertag• Head of Development @ Team Internet• Online-/Tech-Business > 15 years• AWS customer since 2008• Co-Organizer AWS Usergroup Munich

Team Internet

Team Internet

Team Internet

Team Internet

Team Internet

Team Internet

Team Internet @ AWS• We try to

– leverage everything– be as efficient as possible– find creative solutions to roadblocks– save money whenever possible

EC2• > 100 Reserved Instances• Nearly no need to scale down

• Bare metal in co-location or own computing center?

EC2• Need to scale up quickly• Transparent redundancy (AZ)• Networking (VPC, Firewalls, Routing)• Traffic costs• Bandwidth per AZ

Services – “I can do that by myself!”• Know-How• Development (Improvement, Bugfixing)• High Availability• Operations (24/7)

Additional Features of AWS• Get features for free:

– API, SDKs, CLI– Global footprint (low latency)– Monitoring (Cloudwatch)– IAM– High Availability

Flexibility – we need it!• Testing, Staging...• Find (hardware) bottlenecks• Add technologies to our stack

• Try out new/crazy ideas and save money if they really work!

MongoDB vs. DynamoDB• MongoDB-Cluster:

– >5 TB– >300 GB every day– rotate DB every week (~2 TB)

MongoDB vs. DynamoDB• Several issues with MongoDB

– MongoDB updates = adventure– Maintenance sometimes difficult and time consuming– More data = more trouble

MongoDB vs. DynamoDB• DynamoDB:

– Managed = no maintenance– Updates = not our/no problem– More data = not our/no problem

MongoDB vs. DynamoDB• DynamoDB as a replacement

– Rotate Table/DB every day (scripted)– Automatic scaling = save money– Lower/more stable response times (< 5ms)

Why no actual numbers?• We don‘t compare apples to oranges • We choose a philosophy: static (bare-metal) vs. flexible (cloud), depending on

the use-case• We reduce costs in the cloud as much as possible but stick to our decision

• Mail: markus@teaminternet.de• Xing: http://xing.to/mo• Linkedin: http://de.linkedin.com/in/ostertag

4: Use Spot Instances

Spot Instance rules

• Price based on supply/demand

• You choose your maximum price/hour

• Your instance is started if the Spot price is lower

• Your instance is terminated if

the Spot price is higher, with 2 minutes notice

• But: You did plan for fault tolerance, didn’t you?

Spot Instance example

On-Demand:

$0.24

$0.028 (11.7%) $0.026 (10,8%)

$3.28

(1367%)

Spot Instances recap• Very dynamic pricing

• Opportunity to save 80-90% cost– But there are risks

• Different prices per AZ

• Leverage Auto Scaling!– One group with Spot Instances

– One group with On-Demand

– Get the best of both worlds

• New: Spot fleets – Manage thousands

of Spot Instance with one API call

5: Use Amazon S3 storage classes

• Reduced redundancy storage class– 99.99% durability vs. 99.999999999%

– Up to 20% savings

– Everything that is easy to reproduce

– Use Amazon SNS lost object notifications

• Amazon Glacier storage class– Same 99.999999999% durability

– 3 to 5 hours restore time

– Up to 64% savings

– Archiving, long-term backups, and old data

• Use life-cycle rules

6. Optimize Amazon DynamoDB

capacity units

• Read/write capacity units (CUs) determine

most of DynamoDB cost

• By optimizing CUs, you can save a lot of money

• But:– Need to provision enough capacity to not run into capacity errors

– Need to prepare for peaks

– Need to constantly monitor/adjust

• Use caching to save read capacity units– Local RAM caches at app server instances

– Check out Amazon ElastiCache

• Think of strategies for optimizing CU use– Use multiple tables to support varied access patterns

– Understand access patterns for time series data

– Compress large attribute values

• Use Amazon SQS to buffer over-capacity writes

Amazon SQS can buffer requests

Dynamic DynamoDB

DynamoDB optimization example

Caching/Optimization:

80% saved

Cache

flush

Dynamic

DynamoDB:

20% saved

Growth +

new features

7. Offload your architecture

• The more you can offload, the less

infrastructure you need to maintain, scale,

and pay for

• Three easy ways to offload:– Use Amazon CloudFront

– Introduce caching

– Leverage existing Amazon web services

Offload popular traffic to

Amazon S3 and CloudFront

Offload databases through caching

Leverage existing services• Amazon RDS, Amazon DynamoDB or Amazon

ElastiCache for Redis, Amazon Redshift– Instead of running your own database

• Amazon CloudSearch– Instead of running your own search engine

• Amazon Elastic Transcoder

• Amazon Elastic MapReduce

• Amazon Cognito, Amazon SQS, Amazon SNS,

Amazon Simple Workflow Service, Amazon SES,

Amazon Kinesis, and more …

Simple, more reliable, lower cost

Cost monitoring

&

analysis

AWS TCO calculator

AWS simple monthly calculator

AWS Billing Console

AWS Cost Explorer

AWS billing alerts

Let’s recap

1. Turn off unused instances

2. Use Auto Scaling

3. Use Reserved Instances

4. Use Spot Instances

5. Leverage Amazon S3 storage classes

6. Optimize Amazon DynamoDB capacity units

7. Offload your architecture

… and remember to iterate!

Constantin Gonzalez

glez@amazon.de

@zalez