How to squeeze AWS costs

Jacek MigdałWarsaw, AWS User Group, 2015-09-24

Who am I? The Cloud Juggler!

Senior Software Engineer at

Tripled engineering headcount in Europe!

We do logs in cloud:

● supercharged grep | awk in the browser● Google for your internal infrastructure

Many TB / day / client

AWS ~= Industrial revolution

Side effect: cost ~= pollution

AWS bill. Engineering productivity or play?

Challenge #1: Know your resources

Run and forget.

Problem both in development and production.

Solution #1: Know your resources

Separate dev and prod:

● Separate AWS accounts.● Consolidated billing.● Simple to monitor and delete resources.● Additional benefits: isolation and

security.

Solution #1: Know your resources

Monitor proactively:

● Tag and name resources in production.● Setup monitoring rules.● Get alert if you go over the limit.● Either AWS or 3rd party (CloudHealth).

Solution #1: Know your resources

Shutdown by default:

● Terminate development resources after business hours.

● Notify by email.● Need to mark them to persist them.

Challenge #2: Server capacity planning

Majority of the costs are EC2.

Some planning is needed to be cost efficient.

Solution #2: Server capacity planning

Best way to run machines:

● Commit for a year, get 40% discount -> Reserved Instance.

● Use auto-scaling and spot.● Replace with S3, DynamoDB, ...

Solution #2: Server capacity planning

Determine limiting metric and pick:

● Instance family (c3, m3, r3, …).● Right-size instance type and cluster size.

Initially guess than benchmark.

Optimize the code!

Challenge #3: I have 100 items on my plate.

We are too busy and cost is yet another thing.

We don’t have time to do everything.

Solution #3: I have 100 items on my plate.

Rhythm:

● Buy RI regularly (e.g. monthly).● Inspect aws resources regularly.● Setup cost alerts at different levels.

Solution #3: I have 100 items on my plate.

Standardize and simplify:

● Instance types: fewer is easier for RI.● Divide system into smaller parts by tags.● Single owner with fallback.

Solution #3: I have 100 items on my plate.

Define and measure:

● usage metrics: e.g. page views● potential optimizations impact vs. effort

If we have data, we will make the right decision.

If we have opinions, let's go with mine.

Thank you

Images

1. Public domain: https://pl.wikipedia.org/wiki/Rewolucja_przemys%C5%82owa#/media/File:Hartmann_Maschinenhalle_1868_(01).jpg

2. Public domain:

https://en.wikipedia.org/wiki/Widnes#/media/File:Widnes_Smoke.jpg

3. Creative Commons:https://commons.wikimedia.org/wiki/File:Money_Cash.jpg