Leveraging AWS for Lean Development Lifecycle
Ryan Shuttleworth – Technical Evangelist @ryanAWS
What is the “lean development cycle”?
Fast
Flexible Pivot
Iterate Agility
Code not documentation
MVP Sprint Working software
Continuous integration
Idea Product
Idea Product
Least friction
Highest quality
Idea Product
A Startup Journey (your experiences may vary, but the
following is based on real experiences…)
“Democracy is the art of running the circus from
the monkey cage.”
H. L. Mencken (1880 - 1956)
“Democracy is the art of running the circus from
the monkey cage.”
H. L. Mencken (1880 - 1956)
Outside it’s nice and neat: Have a picnic on the lawn at the zoo
Inside it’s chaos: Screaming, swinging, banana-skins
“Democracy is the art of running the circus from
the monkey cage.”
H. L. Mencken (1880 - 1956)
Outside it’s nice and neat: Have a picnic on the lawn at the zoo
Inside it’s chaos: Screaming, swinging, banana-skins
Sound like a startup? Let everyone feel safe and comfortable looking at you from the outside
“Democracy is the art of running the circus from
the monkey cage.”
H. L. Mencken (1880 - 1956)
Outside it’s nice and neat: Have a picnic on the lawn at the zoo
Inside it’s chaos: Screaming, swinging, banana-skins
Sound like a startup? Let everyone feel safe and comfortable looking at you from the outside Manage the chaos on the inside
“Democracy is the art of running the circus from
the startup monkey cage.”
H. L. Mencken (1880 - 1956)
“Democracy is the art of running the circus from
the startup monkey cage.” (and why the AWS cloud is the perfect place for one)
The idea… 1 You've got a cracking idea, but don't know if there is point or a market
The idea… 1 You've got a cracking idea, but don't know if there is point or a market
1x free tier instance Zero cost
What
AWS
Take-away
CloudFormation a Wordpress site Go buy a $30 template Make it look real
Wait and see what traffic/interest you get
The idea… 1 You've got a cracking idea, but don't know if there is point or a market
http://aws.amazon.com/cloudformation/aws-cloudformation-templates/
2 Analytics prove you have a point, web hits increase traffic. A lot.
Proving it…
2 Analytics prove you have a point, web hits increase traffic. A lot.
CDN your marketing presence Easy, 60 minute update You look global
What
AWS
Take-away You now have interest Keep it ticking over Define the Minimum Viable Product (you can get away with)
Proving it…
They came, but then they go because it's too slow to respond They came, signed up for a beta, proving you might actually need to build something…
3 You’ve got some people to give something to, so give them something
Customers…
3 You’ve got some people to give something to, so give them something
Beanstalk – sow the seed, don’t worry about complex operations yet Multi-AZ – small instances, but highly available
What
AWS
Take-away
First phase application – make sure you launch it on a positive SLA Features may be lacking, but it needs to be up!
Deliver something simple, deliver it well Keep those early adopters happy
Customers…
3 You’ve got some people to give something to, so give them something
Beanstalk – sow the seed, don’t worry about complex operations yet Multi-AZ – small instances, but highly available
What
AWS
Take-away
First phase application – make sure you launch it on a positive SLA Features may be lacking, but it needs to be up!
Deliver something simple, deliver it well Keep those early adopters happy
Customers…
Application package is deployed into Beanstalk
Elastic Beanstalk
CloudFormation
Tools
User Application
Application Service
HTTP Service
Language Interpreter
Operating System
Host
Which creates the container in EC2
Elastic Beanstalk
CloudFormation
Tools
Beanstalk takes care of the environment…
Elastic Beanstalk
CloudFormation
Tools
…adding Elastic Load Balancer…
Elastic Beanstalk
CloudFormation
Tools
…Auto Scaling groups
Elastic Beanstalk
CloudFormation
Tools
…and launching instances
Elastic Beanstalk
CloudFormation
Tools
…all wired into the Beanstalk framework
Elastic Beanstalk
CloudFormation
Tools
…and published under a CNAME
Elastic Beanstalk
CloudFormation
Tools
…with logs and app versions held in S3
Elastic Beanstalk
CloudFormation
Tools
4 Inevitably, 3am on a Saturday night, things start to break….
Stuff breaks….
4 Inevitably, 3am on a Saturday night, things start to break….
You started on small instances – up them. You’ve plenty to play with
What
AWS
Take-away
Your database grew – you haven’t optimised it Your application is naïve – it doesn’t cache anything
Engineering doesn’t happen overnight With AWS you bought some time – 4-6wks on the clock to make life easy again
Stuff breaks….
5 Got customers, demand is growing and now have functionality to deliver
Survival…
5 Got customers, demand is growing and now have functionality to deliver
Reserve instances – reduce some cost now Build dev-test disposable environments - Need to test things before we deliver (Beanstalk, CloudFormation)
What
AWS
Take-away
Optimise the data layer Size instance types properly
We bought time to pay down technical debt Reduced costs, started to act like a real outfit
Survival…
6 Cold sweats at 2.30am – service outage – we could have lost everything!
Resilience…
6 Cold sweats at 2.30am – service outage – we could have lost everything!
RDS resiliency S3 snapshots and EBS/AMI management
What
AWS
Take-away
How resilient is my RDS setup? What are my snapshot strategies? What is my point in time recovery period for RDS?
Few clicks (and a little more cost) and it’s taken care of 35 days of backup history on RDS Slave failover - automatic
Resilience…
7 You have customers from the real world, and they are EVERYWHERE
Scale…
7 You have customers from the real world, and they are EVERYWHERE
CDN static content served from application CMSs CDN dynamic content where you can, play with very low TTLs Review Azs, ASGs for origin servers
What
AWS
Take-away
Now your application is your bottleneck App hits are very high, database is fine Do the bare minimum of heavy lifting in your application
Push load away from your core application Again, buy time (spend more time on features) Keeping reviewing so you don’t accrue too much technical debt
Scale…
9 You grew your service, now focus on ‘supportability ratio’
Automate…
9 You grew your service, now focus on ‘supportability ratio’
CLI, APIs CloudFormation, CloudInit
What
AWS
Take-away
More customers to less staff is better – the ratio Focus on tools and automation to keep your team lean Build a DevOps team (you have one by now anyway!)
No more 2.30am problems DevOps from iPhone (on a Friday night at the pub)
Automate…