developers guide to aws

8/13/2019 Developers Guide to Aws

1/20

The Develope GTO AMAZON WEBSERVICES


2/20

1Guide to Amazon Web Services

1 What is Infrastructure as a Service?

2 How to Determine if IaaS is right for you2 IaaS may not be right for you if...

3 The Major IaaS Players

4 Pro Tips for IaaS Evaluation

5 5 Best Practices for Deploying an Application on AWS

1Guide to Amazon Web Services

1 What is Infrastructure as a Service?

2 How to Determine if IaaS is right for you

2 How to determine if IaaS may not be right for you

3 The Major IaaS Players

4 Pro Tips for IaaS Evaluation

5 5 Best Practices for Deploying an Application on AWS

4AWS Outage Survival Guide

6 Surviving AWS Failures with a Node.js & MongoDB Stack

10 Migrating from GoDaddy DNS to Amazon Route 53

11 Recovering from a DNS service outage in AWS using Monit

Table of Content

i
http://www.kinvey.com/


3/20

Chapte 1Guide to Amazon Web Services

1
http://www.kinvey.com/http://en.wikipedia.org/wiki/Software_as_a_servicehttp://en.wikipedia.org/wiki/Platform_as_a_servicehttp://searchcloudcomputing.techtarget.com/definition/Infrastructure-as-a-Service-IaaShttp://social.technet.microsoft.com/wiki/contents/articles/4633.what-is-infrastructure-as-a-service.aspx#Infrastrcuture_as_a_Servicehttp://searchcloudcomputing.techtarget.com/definition/Infrastructure-as-a-Service-IaaShttp://www.kinvey.com/blog


4/20

Welcome to the latest Kinvey eBook. We

aim to help our community of develop-

ers keep pace with the latest trends in

app development, tools and marketing.Typically we share our perspective on

our blog , but when our audience is

interested in a topic that cant be done

justice in 500 words, we publish an

eBook instead.

This eBook curates some of the best

thinking from Kinvey and outside

experts on how developers should

approach Infrastructure as a Service

(IaaS). The eBook emphasizes Amazon

Web Services (AWS) because its the best

known vendor in the space, though we

dont recommend one provider over

another. The eBook highlights the

benets of IaaS, helps in the IaaS vendor

selection process, shares best practices

for hosting an app on AWS, and provides

tips for what to do in case of a service

outage.

What is Infrastructure as a Service?

There are a handful of __ as a service

(*aaS) providers comprising the mobile

and cloud computing ecosystems today.

Although individually each company may

be unique, collectively they share acommon goal: accelerating the rate of

innovation by removing costs and

barriers to technology deployment. IaaS

2

DEVELOPERS GUIDE TO AMAZON WEB SERVICES

is perhaps the most fundamental of the

categories. Before diving into our guide,

lets rst review IaaS basics:

According to TechTarget, Infrastructure

as a Service is a cloud-based provision

model that services cloud storage, virtual

servers and networking components to

application owners for a usage-based

cost. Its goal is to is to become a

foundation for Platform as a Service

(PaaS) and Software as a Service (SaaS)

providers by providing a exible

operating environment. In the IaaS

environment, the service provider owns,

runs and maintains the infrastructure

equipment, while the consumer takes

responsibility for conguration and

operations of the guest operating

system, software and database.

A variety of technologies can benet

from IaaS: cloud-based CRM systems,

web, media and mobile applications, big

data systems, and much more. But, IaaS

is not right for everyone. There are some

important factors to consider before

determining if your application would

benet from IaaS.


5/20

Chapte 2How to Determine if IaaS is rightfor you

3

+
http://www.kinvey.com/http://www.zdnet.com/blog/service-oriented/saas-paas-and-iaas-three-cloud-models-three-very-different-risks/8815http://www.quora.com/Web-Development/How-does-one-determine-proper-infrastructure-selection-for-a-web-app-on-AWShttp://www.informationweek.com/hardware/utility-ondemand/iaas-a-bad-deal-not-so-fast/232901031


6/20

There are pros and consassociated with IaaS

that must be assessed...

Choosing an IaaS provider is a big

decision. You want to trust the provider

with your data and, essentially, yourentire application infrastructure. As with

any service, there are pros and cons

associated with IaaS that must be

assesse d before deci ding whether or not

to use it for your application.

You could benet from IaaS if...

there is the potential for spikes in users

or usage. Do you have upcoming press

coverage th at may cause spikes in

downloads? Is your application

seasonal?

you plan to expand your feature set.

With an increase in features comes an

increased demand on infrastructure.

Scalability doesnt only apply to the

number of users, sometimes features

too need to scale.

youre an individual developer, small dev

shop or startup with no existing data

center infrastructure or youre an

established company taking on a large

project that would require signicant

additional data center infrastructure or

staff.

you have a low server-to-admin ratio

and are looking to cut costs.


IaaS may not be right for you if... usage is minimal or at, and you have

no plans to drive signicant growth.Maybe your product is built specically

for only a small subset of users (e .g., an

internal application for a small company).

you dont have a clear sense of your

applications storage and networking

needs. Successful IaaS deployments

benet from clear user up-front

requirements.

you are an enterprise concerned about

rogue users. (One risk of IaaS is rogue or

unwarranted commandeering of

services. Because IaaS requires gover-

nance and usage monitoring, Vordel's

Mark O'Neill recommends that

enterprises establish cloud service

governance frameworks that helpprevent employees from accessing

information or services they are not

permitted to use.)

4
http://www.infoworld.com/d/cloud-computing/how-choose-iaas-provider-198317?page=0,0http://i.techrepublic.com.com/blogs/cloudproms.png?tag=content;siu-containerhttp://i.techrepublic.com.com/blogs/cloudproms.png?tag=content;siu-containerhttp://www.techrepublic.com/blog/datacenter/side-by-side-comparisons-of-iaas-service-providers/5717http://i.techrepublic.com.com/blogs/userconcerns1.png?tag=content;siu-containerhttp://en.wikipedia.org/wiki/Platform_as_a_servicehttp://www.kinvey.com/http://en.wikipedia.org/wiki/Backend_as_a_servicehttp://en.wikipedia.org/wiki/Platform_as_a_servicehttp://i.techrepublic.com.com/blogs/userconcerns1.png?tag=content;siu-containerhttp://www.techrepublic.com/blog/datacenter/side-by-side-comparisons-of-iaas-service-providers/5717http://i.techrepublic.com.com/blogs/cloudproms.png?tag=content;siu-containerhttp://www.infoworld.com/d/cloud-computing/how-choose-iaas-provider-198317?page=0,0


7/20

5

The Major IaaS Players: AComparison

Once youve decided to use an IaaSsolution, its time to pick a vendor. While

AWS may be the most prevalent player

in the space - it is estimated to own

roughly 70 percent of the IaaS market -

there are several other reputable

vendors to consider, and many factors to

evaluate. Rest assured, there are plentyof resources out there to help you

narrow down the options.

This char t from TechRepublic evaluates

the best-known IaaS providers against

how they compare to common cloud

promises. The comparison takes into

account a range of factors, including

pricing (variety, average, data transfer


and storage costs), scalability (scaling up

and down, monitoring, and APIs), and

choice / exibility (number of data

center locations, number of instance

types, and supported operating

systems).

Another way to size-up vendors is bycommon user concerns, which is the

thrust of this second TechRepublic chart .

Specically, the table assesses vendors

against security features (certications

and protection), ease of migration (open

standards and VM upload), and

reliability [service age, Service LevelAgreement (SLA), and support].

Pro Tips for IaaS EvaluationRegardless of whether you select an IaaS

vendor based on its cloud promises or

user concerns, Kinveys lead architect,

Shubhang Mani, advises you consider

the following:

Is IaaS really the solution you need?

Depending on the complexity of yourinfrastructure needs, you might be

better off opting for a higher layer in the

stack such a s a Platform as a Service

(PaaS), or even Backend as a Service

Beyond data, developerswant to tap into manytools and services thatother clever minds have

created...

+
http://www.slideshare.net/AmazonWebServices/aws-101-cloud-computing-seminar-2012#btnNext%20%E2%80%93http://www.kinvey.com/http://www.slideshare.net/AmazonWebServices/aws-101-cloud-computing-seminar-2012#btnNext%20%E2%80%93


8/20

(BaaS). The advantages of choosing

these alternatives are reduced opera-

tional complexity and decreased time to

market. The disadvantag es are re ducedcontrol and a narrower set of choices as

to the underlying infrastructure

components. PaaS or BaaS might be a

good starting point, allowing you to

focus on building the application /

system until you deem it necessary to

exert greater control on the choice ofinfrastructure com ponents.

IaaS is not a silver bullet

Some key benets of IaaS include ease in

deployment, redundancy, and the ability

to scale much faster than conventional

means. Deploying a distributed system,

especially across geographic regions, can

also be achieved in more easily.

However, it is i mportant to note that

while IaaS gives you the means to

provision, deploy and scale infrastruc-

ture, it is up to you to congure, monitor

and maintain said infrastructure and use

it in a manner that makes the most

sense to your system. You might be able

to build multiple redundancies into a

complex system sitting across geograph-

ic regions in a matter of hours, but ifyour rewall rules are improperly

congured, youre still vulnerable to

attack.

6


Plan for the worst case

No system is infallible. Outages can

occur, even at your cloud provider of

choice. It is important that you have aplan in place to address this were it to

occur. For example, you should consider

offsite backups for your critical data and

alternate deployments from your main

location.

Understand your support requirementsNot all IaaS providers provide the same

levels of support. In some case s, support

is priced independently from the actual

product and may end up being quite

expensive. Most providers offer a free

support level via community forums.

This may or may not be sufficient

depending on your needs.

With so many factors to consider,

selecting an IaaS vendor for your app is

arguably the most difficult part of the

process. The next step is actually

deploying your app on the chosen IaaS

provider. Because AWS is the most

popular vendor it will be the main focus

moving forward.

5 best practices for deploying an

application on AWSAmazon Web Services is a major

Infrastructure as a Service provider that

provides elastic capacity, quick deploy-

ment and auto mation for applicatio ns
http://aws.amazon.com/iphone-application-hosting//http://en.wikipedia.org/wiki/Platform_as_a_servicehttp://www.backendasaservice.com/http://www.backendasaservice.com/http://www.sdtimes.com/blog/post/2011/10/21/Best-practices-for-AWS.aspxhttp://www.quora.com/Web-Development/How-does-one-determine-proper-infrastructure-selection-for-a-web-app-on-AWShttp://aws.amazon.com/free/http://www.sdtimes.com/blog/post/2011/10/21/Best-practices-for-AWS.aspxhttp://www.kinvey.com/http://www.sdtimes.com/blog/post/2011/10/21/Best-practices-for-AWS.aspxhttp://aws.amazon.com/free/http://www.quora.com/Web-Development/How-does-one-determine-proper-infrastructure-selection-for-a-web-app-on-AWShttp://www.sdtimes.com/blog/post/2011/10/21/Best-practices-for-AWS.aspxhttp://en.wikipedia.org/wiki/Software_as_a_servicehttp://www.backendasaservice.com/http://en.wikipedia.org/wiki/Platform_as_a_servicehttp://en.wikipedia.org/wiki/Platform_as_a_servicehttp://aws.amazon.com/iphone-application-hosting//


9/20

7

without using capital expenditure.

Within AWS are several infrastructure

building blocks, including EC2, S3, and

RDS, to name a few. Click here for a fulllist of AWS products for mobile applica-

tion hosting.

AWS may be a compelling choice for

your apps infrastructure needs, but

remember, if you want more than just

hosting, there are other categories of*aaS vendors, such as Platform as a

Service , Software as a Service , and

Backend as a Service , that address

different application needs. You may

want to consider vendors in adjacent

service categories as well. That said,

below is a list of best practices for

hosting an app on AWS.

Use the right tool for the job.Alex Handy , Senior editor of SD Times,

advises: Know what your project/appli-

cation is and the problem it solves

before you dig in. Let AWS manage the

infrastructure so you can focus on the

business you do best. As previously

mentioned, there are several services

within the AWS offering. Alex suggests

combining multiple: For example, try

Amazon Relational Database Service for

your database, AWS Elastic Beanstalk for

your development environment, or

Amazon Elastic Map Reduce for your

Hadoop cluster and Big Data needs.


Familiarize yourself with these tools

before diving in.

Start smallStart by moving a small project to AWS

before your full project is underway.

This way you can fully test and learn

about the various components that

youll be using without worrying about

managing an entire project.

Start free.

Consider starting with Amazons free tier

to test and become familiar with the

Know what yourproject/appliccation is

and the problem it solvesbefore you dig in.

platform before jumping into a full

development effort. You get 5GB of

storage free for a year on S3, so you can

easily back something up for free to see

if AWS is the way to go.

Leverage multiple availabilityzones.If you want your app to be fault-tolerant,

mirroring across availability zo nes is key
http://www.slideshare.net/AmazonWebServices/building-powerful-web-applications-in-the-aws-cloud-a-love-story-jinesh-varia#btnNexthttp://www.kinvey.com/http://www.kinvey.com/http://www.kinvey.com/http://www.slideshare.net/AmazonWebServices/building-powerful-web-applications-in-the-aws-cloud-a-love-story-jinesh-varia#btnNext


10/20

8

for high availability and disaster

recovery. Ensure your design anticipates

and manages component failure to

signicantly reduce the chances of itfailing.


...always have a backupplan in the event of an

outage

Designfor failure and nothing will failUnderstand Amazons disaster recovery

principles, and always have a backup plan

in the event of an outage. Our developers

have compiled multiple guides on recover-

ing from and preparing for an Amazon

outage - take a look at the next segment,

and learn how to be proactive for the sake

of your application.

AWS outage survival guideAn outage on your cloud service provider

likely means an outage for your app and

thus a delay in the experience on the

user-end. This can be extremely detrimen-

tal to your apps ratings and overall usage.

The unfortunate truth is that everyone has

unforeseen outages due to factors out of

our control. If it wasnt AWS, it would be

the other cloud provider you chose. But

there is still hope: when AWS is down,you can still be up if you take the proper

measures to prepare for possible

outages in advance.

4 things you didnt know about AWS

outages

The eastern region of the US is the

most likely area to suffer outages

because ... it has the oldest data centers.

it is the largest in terms of data center

footprint.

it is the default region for most

customers, who dont bother changing itbecause its cheaper and/or they arent

aware they can change the region .

Availability zones are guaranteed to

be distinct per customer only.

However, there are no guarantees as to

the composition of the availability zoneacross customers. To illustrate, if you

launch an instance in availability zone 1a

and a different customer launches an

instance in their availability zone 1a, the

two instances are not guaranteed to run

on the same subset of physical

infrastructure. This is why Amazon doesnot specify or call out names of availabil-

ity zones in their outage status updates,

because 1a for one person could be 1d

for another.


11/20

9

Some outages are worse than others.

This is because outages that affect core

infrastructure components such as

Amazon EBS (Elastic Block Store) have aripple effect on other AWS products that

utilize these components. For example,

it is possible to provision an EC2

instance using ephemeral storage that is

bound to the instance. This instance

does not utilize EBS and should

theoretically be impacted to a lesser

degree. Lets say you have an application

that runs on this instance. This applica-

tion also uses a MySQL database and

youve opted to use Amazons Relational

Database Service (RDS). Now youre back

to (potentially) being affected by an EBS

outage since RDS uses EBS for storage.

Amazon provides a rich set of APIs

that allow users to access the control

plane for various infrastructure

components.

In fact, a judicious use of these APIs via

client libraries and scripts is what allows

one to be able to do things like launch

new instances based on traffic volume

and / or provision new infrastructure as

necessary. When an outage occurs,

affected customers attempt to remedi -ate their situation by trying to provision

new servers in other availability zones

and / or regions in order to redirect web

traffic and/or proceed with other tasks.


control planes unusable, especially if the

outage affects components that are

used to service the control planes

themselves. This results in customers

being unable to minimize downtimefaced by their application / systems. A

potential solution to this would be to

have a hot / cold standby set of compo-

nents ready in a different region. When

the outage occurs, it would be a matter

of bringing these components online

with the understanding that outages

that span regions are far less likely than

those within a single region.

This means that theres a fairly large

spike in traffic hitting the control plane

which invariably results in slowdowns

and timeouts. Another side effect of the

outage could also be to render the


12/20

SURVIVAL KITAWS

Chapte 3 Surviving AWS Failures with anode.js & mongoDB Stack

10
http://www.kinvey.com/http://aws.amazon.com/elasticloadbalancing/http://www.mongodb.org/display/DOCS/Replica+Set+Tutorialhttp://aws.amazon.com/architecture/http://www.kinvey.com/blog/104/surviving-aws-failures-with-a-nodejs-and-mongodb-stack


13/20

11

In this segment , well explain how to fully

prepare for an AWS EC2 outage with a

node.js and MongoDB stack. Node+Mon-

go on EC2 is a very popular softwarestack among web services developers.

There are many user guides on how to

design this system with built-in redun-

dancy so that even coordinated failures

dont bring down the service. The

absolute minimum for a resilient service

requires a MongoDB replica set behind aload-balanced node farm.

You are not ready for an EC2 outage until

you have deliberately shut down

components in your system and veried

the expected behavior. As you periodical-

ly do this, you might discover that there

are gaps you did not account for. Take

the following steps to be as prepared as

possible:

1. A Node.js single event will by default

crash on an unhandled exception. Use

upstart or forever to restart the process.

2. Use Monit, an external process on your

server that makes liveness checks and

potentially restarts your service. Monit

will also email you if and when it had to

restart. While upstart ensures that yourprocess is up, monit ensures that it is

responsive.

3. Your application instances and your


MongoDB instances should each be load

balanced across multiple Availability

Zones. The more the better.

4. Place the node servers and the mongo

servers all in a security group, which

allows only the Mongo ports internally

and your application ports externally.

This is trivial to set up and protects your

database from external requests.

5. MongoDBs authentication provides

additional protection. Mongos security

model has limited robustness, but

having authentication in your MongoDB

store is still useful even if the application

and the database are inside an EC2

security group. For your data to get

exposed, you will have to make multiple

mistakes at the same time, which

happens, but the chances are greatly

reduced.

6. Ensure that failover happens smoothly.

Shutdown the primary Mongo instance

and see what happens as requests keep

coming in. The replicas notice the down

primary and one of them takes over, but

upon an incoming request you see this

error message: unauthorized db:mydb

lock type:-1 client:127.0.0.1

7. What this error message means is that

the failover happened, but your

applications request is not authenticat-


14/20


15/20

13

your domain name from the list, and

click Go to Record Sets in the top right.

Youll see that Route 53 populated your

domain with a set of NS records and

SOA records.

5. Your current site may have extra DNS

records needed. For example, if you host

a blog on Tumblr, you probably have a

CNAME record setup to create that link.

Also, if you receive email at your domain,you probably have an MX record set for

that. Make a list of all these extra

records youll need, and add them now

(use the Create Record Set button).

6. Finally, its time to make the switch.

Head over to your current registar, and

change the nameservers from the

GoDaddy addresses to the ones

provided in the NS record set on your

Route 53 dashboard. Note: because DNS

servers around the world cache this

value, it may take some time to see the

change work while the update propa-

gates through the DNS system.

Recovering from a DNS service outagein AWS using MonitOur nal AWS outage survival guide uses

a tool called Monit to recover from a

DNS service outage. A DNS failure isnt

something you see everyday at AWS, but

when it happens it can cause problems


with your application. If your app relies

on DNS to connect to another server (i.e.

database), it may stop trying after a few

failed lookups. At this point things

probably require manual intervention,

even if the DNS service recovers. The key

takeaway from this is that you should

know how your application fails when a

network service like DNS is unavailable.

And you should know whether or not it

will require manual intervention torecover.

To prevent manual intervention, you can

use a tool called Monit. Monit is a

daemon that is responsible for monitor-

ing your server and application health. It

can also be congured to check the

health of external services such as DNS.

Lets take a look at a basic cong to start:

This cong monitors an HTTP application

that listens on 127.0.0.1 on port 9009.

When there is a failure, Monit alerts you

and attempts to restart the process.

We dont want to constantly restart the

service if it is unhealthy, so if the app

fails 10 times within 10 cycles, then

Monit will leave the app off and stop
https://gist.github.com/3996208https://gist.github.com/3996208https://gist.github.com/3996208https://gist.github.com/3996273https://gist.github.com/3996273https://gist.github.com/3996273https://gist.github.com/3996273http://www.kinvey.com/https://gist.github.com/3996273https://gist.github.com/3996208


16/20

14

monitoring it.

When a network service fails like DNS,

your application becomes unhealthy,tries to restart 10 times, and then

becomes unmonitored. Now youre at a

point w here things require manual

intervention to recover, even if DNS

becomes available. Lets congure Monit

to take care of everything for us.

Building on our previous conguration:

- Were still monitoring our HTTP app on

port 9009

- When the application fails now, it kicks

off the aws-dns-healthcheck monitor.

- The aws-dns-healthcheck monitor

checks to see if it can resolve DNS on

172.16.0.23 (the default AWS DNS

server).

- When DNS reports healthy for 3 cycles,

monit execs a script to try to recover the

app.

The example recovery script is simple:

It calls /usr/bin/monit start appsrv1

which will attempt to start the app and


re-enables monitoring. It also disables

our aws-dns-healthcheck since we know

DNS is healthy.

You should now be able to recover your

app during a DNS failure. To be 100%

sure this works as intended, we can

simulate another DNS outage using

iptables.

Run this command: sudo iptables -AOUTPUT -p udp dport 53 -j DROP

This command will prevent any DNS

queries from completing. If your app

relies on DNS you should see it fail once

its DNS cache has expired. This should

trigger the DNS check to be enabled via

Monit. You can check the status of Monit

monitoring using the command: sudo

monit status

To re-enable DNS run this:

sudo iptables -F

Your application should now try to

recover after the DNS check returns to a

healthy state.

The examples above focus on how to

approach a failure in the DNS service. In

reality there could be a myriad of

external services that may affect thehealth of your app. You can use the

same approach as described above and

expand the aws-dns-healthcheck into a

generic health check for your applica-
http://mmonit.com/wiki/Monit/ConfigurationExampleshttp://www.kinvey.com/http://mmonit.com/wiki/Monit/ConfigurationExamples


17/2015

tion. This could include testing network

connectivity, connectivity to external

services (e.g: a database), and any other

processes that your app depends on.You can nd examples for monitoring all

kinds of services on the Monit website

here.

Again, the takeaway from this is to know

how your application behaves under

different failure scenarios. Testingconnectivity loss, loss of network

services, and other failures are very

important when building a high

availability applicatio n.

http://www.kinvey.com/about/kellyricehttp://www.linkedin.com/profile/view?id=12417476&locale=en_US&trk=tyahhttp://www.kinvey.com/about/davewasmerhttp://www.kinvey.com/about/ivandstoyanovhttp://www.kinvey.com/about/jakemckibbenhttp://www.kinvey.com/about/shubhangmanihttp://www.kinvey.com/about/kellyrice


18/20

Written by

Kelly Rice and Shubhang Mani

Designed by

Jake McKibben

Survival Guide Authors

Ivan Stoyanov, Dave Wasmer

and Joey Imbasciano


19/20


20/20

developers guide to aws

Documents