the lean cloud for startups with aws - introduction & aws overview
TRANSCRIPT
The Lean Cloud for Startups with AWS
aws.amazon.com/free aws.amazon.com/careers/
A bit of background Introduction & AWS overview
Consumer Business
Tens of millions of active customer
accounts
Eight countries: US, UK, Germany,
Japan, France, Canada, China, Italy
Seller Business
Sell on Amazon websites
Use Amazon technology for your own retail website
Leverage Amazon’s massive fulfillment
center network
IT Infrastructure Business
Cloud computing infrastructure for hosting web-scale
solutions
Hundreds of thousands of
registered customers in over 190 countries
Deep experience in building
and operating global web
scale systems
About Amazon Web Services
?
…get into cloud computing?
How did Amazon…
Over 10 years in the making
Enablement of sellers on Amazon
Internal need for scalable deployment environment
Early forays proved developers were hungry for more
AWS Mission
Enable businesses and developers to use web services* to build scalable,
sophisticated applications.
*What people now call “the cloud”
Not excess capacity!
We’ve been doing big for a while…
Each day AWS adds the equivalent server capacity to power Amazon when it was a global, $2.76B
enterprise
(circa 2000)
And it keeps getting bigger
0.000
250.000
500.000
750.000
1000.000
1 Trillion
750k+ peak transactions per second
Objects in S3
Customer focus
May 2012: Amazon Elastic MapReduce Now Supports Hive 0.8.1 Amazon RDS Announces Oracle Enterprise Manager Support Announcing VM Export for Amazon EC2 AWS Console Enhancements for Elastic Load Balancing: Listener, Certificate, and Cipher Management Amazon RDS announces support for MySQL Read Replica in Amazon VPC AWS Elastic Beanstalk Now Available in the EU (Ireland) Region Amazon Simple Email Service Announces Domain Verification Amazon CloudFront Now Supports Dynamic Content Monitor Your AWS Charges with Billing Alerts Using Amazon CloudWatch Announcing API and AWS Identity & Access Management Support for AWS Storage Gateway New Managed Services for Windows Developers Worldwide New and Updated Microsoft SQL Server Offerings on Amazon EC2 New Amazon RDS for Oracle Capabilities and Multi-AZ Enhancements
June 2012: AWS Elastic Beanstalk Announces Updated Command Line Interface Amazon SES Announces Bounce and Complaint Notifications Cluster Compute Eight Extra Large Instance Type Now Available in EU-West AWS CloudFormation Supports Amazon DynamoDB and Amazon CloudFront Dynamic Content New Sydney, Australia Edge Location for Amazon CloudFront & Amazon Route 53 AWS Support Expands Free Tier, Adds New Features, Lowers Prices Amazon Elastic MapReduce Announces Support for HBase Amazon RDS MySQL on t1.micro, starting at just $19 a month Announcing AWS Identity and Access Management (IAM) roles for EC2 instances Announcing Internal Load Balancing in Amazon Virtual Private Cloud Announcing Spot Integration with Auto Scaling and CloudFormation AWS Billing enables enhanced CSV reports and programmatic access Amazon ElastiCache Launches Free Trial Program
April 2012: AWS CloudFormation Support for Creating VPC Resources Amazon DynamoDB Now Available in Three Additional Regions AWS Elastic Beanstalk Now Available in the Asia Pacific (Tokyo) Region Amazon DynamoDB Announces BatchWriteItem Feature Introducing AWS Marketplace AWS Announces The Availability of the Microsoft SharePoint Server on AWS Reference Architecture White Paper AWS Announces CloudSearch Announcing the Availability of Reserved Cache Nodes for Amazon ElastiCache Live Smooth Streaming for Amazon CloudFront
Pace of innovation
May 2012: Amazon Elastic MapReduce Now Supports Hive 0.8.1 Amazon RDS Announces Oracle Enterprise Manager Support Announcing VM Export for Amazon EC2 AWS Console Enhancements for Elastic Load Balancing: Listener, Certificate, and Cipher Management Amazon RDS announces support for MySQL Read Replica in Amazon VPC AWS Elastic Beanstalk Now Available in the EU (Ireland) Region Amazon Simple Email Service Announces Domain Verification Amazon CloudFront Now Supports Dynamic Content Monitor Your AWS Charges with Billing Alerts Using Amazon CloudWatch Announcing API and AWS Identity & Access Management Support for AWS Storage Gateway New Managed Services for Windows Developers Worldwide New and Updated Microsoft SQL Server Offerings on Amazon EC2 New Amazon RDS for Oracle Capabilities and Multi-AZ Enhancements
June 2012: AWS Elastic Beanstalk Announces Updated Command Line Interface Amazon SES Announces Bounce and Complaint Notifications Cluster Compute Eight Extra Large Instance Type Now Available in EU-West AWS CloudFormation Supports Amazon DynamoDB and Amazon CloudFront Dynamic Content New Sydney, Australia Edge Location for Amazon CloudFront & Amazon Route 53 AWS Support Expands Free Tier, Adds New Features, Lowers Prices Amazon Elastic MapReduce Announces Support for HBase Amazon RDS MySQL on t1.micro, starting at just $19 a month Announcing AWS Identity and Access Management (IAM) roles for EC2 instances Announcing Internal Load Balancing in Amazon Virtual Private Cloud Announcing Spot Integration with Auto Scaling and CloudFormation AWS Billing enables enhanced CSV reports and programmatic access Amazon ElastiCache Launches Free Trial Program
April 2012: AWS CloudFormation Support for Creating VPC Resources Amazon DynamoDB Now Available in Three Additional Regions` AWS Elastic Beanstalk Now Available in the Asia Pacific (Tokyo) Region Amazon DynamoDB Announces BatchWriteItem Feature Introducing AWS Marketplace AWS Announces The Availability of the Microsoft SharePoint Server on AWS Reference Architecture White Paper AWS Announces CloudSearch Announcing the Availability of Reserved Cache Nodes for Amazon ElastiCache Live Smooth Streaming for Amazon CloudFront
Pace of innovation
Q2 2012
35 new features
H1 2012
63 2011
82
2010
61
2009
48
2008
24
2007
9 Amazon FPS Red Hat EC2
SimpleDB CloudFront EBS Availability Zones Elastic IPs
Relational Database Service Virtual Private Cloud
Elastic Map Reduce Auto Scaling
Reserved Instances Elastic Load Balancer
Simple Notification Service Route 53 RDS Multi-AZ Singapore Region Identity Access Management Cluster Instances
Elastic Beanstalk Simple Email Service CloudFormation RDS for Oracle ElastiCache
DynamoDB Simple Workflow
CloudSearch Storage Gateway
Route 53 Latency Based Routing
number of released features, sample services described
Keep updated & grab a feed:
aws.typepad.com
What we do in a nutshell
Utility computing
On demand Pay as you go
Uniform Available
On demand Pay as you go
Uniform Available
Utility computing
Utility computing
Compute
Storage
Security Scaling
Database
Networking Monitoring
Messaging
Workflow
DNS
Load Balancing
Backup CDN
On demand Pay as you go
Uniform Available
Utility computing
On a global footprint
Region
US-WEST (N. California) EU-WEST (Ireland)
ASIA PAC (Tokyo)
ASIA PAC (Singapore)
US-WEST (Oregon)
SOUTH AMERICA (Sao Paulo)
US-EAST (Virginia)
GOV CLOUD
Availability Zone
On a global footprint
Edge Locations
Dallas(2)
St.Louis
Miami
Jacksonville Los Angeles (2)
Palo Alto
Seattle
Ashburn(2)
Newark
New York (2)
Dublin
London(2)
Amsterdam
Stockholm
Frankfurt(2)
Paris(2)
Singapore(2)
Hong Kong
Tokyo
Sao Paulo
South Bend
San Jose
Osaka Milan
Sydney
On a global footprint
At the end of a web service
ec2-run-instances ami-b232d0db
--instance-count 3
--availability-zone eu-west-1a
--instance-type m1.small
ec2-run-instances ami-b232d0db
--instance-count 5
--availability-zone eu-west-1c
--instance-type m1.medium
ec2-run-instances ami-b232d0db
--instance-count 2
--availability-zone us-east-1b
--instance-type m1.xlarge
ec2-run-instances ami-b232d0db
--instance-count 2
--availability-zone eu-east-1d
--instance-type m1.xlarge
At the end of a web service
At the end of a web service
elb-create-lb myLoadBalancer
as-create-auto-scaling-group MyGroup
--launch-configuration MyConfig
--availability-zones eu-west-1c
--min-size 2
--max-size 200
ec2-authorize default -p 80
and rich console services
Elasticity
Traditional IT
capacity
Elastic capacity
Capacity
Time Your IT needs
On and Off Fast Growth
Variable peaks Predictable peaks
Elastic capacity
On and Off Fast Growth
Predictable peaks Variable peaks
WASTE
CUSTOMER DISSATISFACTION
Elastic capacity
Elastic cloud capacity
Traditional
IT capacity
Your IT needs
Time
Capacity
Elastic capacity
Fast Growth On and Off
Predictable peaks Variable peaks
Elastic capacity
503 Service Temporarily Unavailable
The server is temporarily unable to service
your request due to maintenance downtime or
capacity problems. Please try again later.
503 Service Temporarily Unavailable
The server is temporarily unable to service
your request due to maintenance downtime or
capacity problems. Please try again later.
From one instance…
…to thousands
And back again…
Time: +00h
<10 cores
Elastic Capacity
Time: +24h
>1500 cores
Elastic Capacity
Time: +72h
<10 cores
Elastic Capacity
Time: +120h
>600 cores
Elastic Capacity
Num
ber
of E
C2 I
nsta
nces
4/12/2008 4/14/2008 4/15/2008 4/16/2008 4/18/2008 4/19/2008 4/20/2008 4/17/2008 4/13/2008
40 servers to 5000 in 3 days
EC2 scaled to peak of 5000 instances
“Techcrunched”
Launch of Facebook modification
Steady state of ~40 instances
The Toolbox
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Reference Model
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Regions An independent collection of AWS resources in a defined
geography
A solid foundation for meeting location-dependent privacy
and compliance requirements
Global infrastructure
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Availability Zones Designed as independent failure zones
Physically separated within a typical metropolitan
region
Global infrastructure
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Global infrastructure
Edge Locations To deliver content to end users with lower latency
A global network of edge locations
Supports global DNS infrastructure (Route53) and
Cloud Front CDN
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Direct Connect Dedicated connection to AWS
VPN Connection Secure internet connection to AWS
Virtual Private Cloud Private, isolated section of the AWS Cloud
Route 53 Highly available and scalable Domain Name System
Networking
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Networking
“We were able to reduce our DNS costs by ninety-three percent, which in tandem allowed us to
shorten our time-to-live (TTLs) for easier, timelier management of
DNS records.”
Nathan Butler The Newsweek/Daily Beast Company
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Vertical Scaling
From $0.02/hr Elastic Compute Cloud (EC2) Basic unit of compute capacity
Range of CPU, memory & local disk options
13 Instance types available, from micro to cluster compute
Feature Details
Flexible Run windows or linux distributions
Scalable Wide range of instance types from micro to cluster compute
Machine Images Configurations can be saved as machine images (AMIs) from which new instances can be created
Full control Full root or administrator rights
Secure Full firewall control via Security Groups
Monitoring Publishes metrics to Cloud Watch
Inexpensive On-demand, Reserved and Spot instance types
VM Import/Export Import and export VM images to transfer configurations in and out of EC2
Compute
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Compute
Auto-scaling Automatic re-sizing of compute clusters based upon demand
Trigger
auto-scaling
policy
Feature Details
Control Define minimum and maximum instance pool sizes and when scaling and cool down occurs
Integrated to CloudWatch Use metrics gathered by CloudWatch to drive scaling
Instance types Run auto scaling for on-demand instances and spot. Compatible with VPC
as-create-auto-scaling-group MyGroup
--launch-configuration MyConfig
--availability-zones eu-west-1a
--min-size 4
--max-size 200
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Compute
Elastic Load Balancing Create highly scalable applications
Distribute load across EC2 instances in multiple
availability zones
Feature Details Auto-scaling Automatically scales to handle request colume
Available Load balance across instances in multiple availability zones
Health checks Automatically checks health of instances and takes them in or out of service
Session stickiness Route requests to the same instance
Secure sockets layer
Supports SSL offload from web and application servers with flexible cipher support
Monitoring Publishes metrics to Cloud Watch
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Storage
S3 - Durable storage, any object 99.999999999% durability of objects
Unlimited storage of objects of any type
Up to 5TB size per object
Feature Details
Flexible object store Buckets act like drives, folder structures within
Access control Granular control over object permissions
Server-side encryption 256bit AES encryption of objects
Multi-part uploads Improved throughput & control
Object versioning Archive old objects and version new ones
Object expiry Automatically remove old objects
Access logging Full audit log of bucket/object actions
Web content hosting Serve content as web site with built in page handling
Notifications Receive notifications on key events
Import/Export Physical device import/export service
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
“Our use of AWS was really a perfect storm of technology coming together with customer
need. AWS allowed us to store information in a cost effective manner while alleviating the
burden of supporting the necessary infrastructure since AWS takes care of that. It really is a win-win for us and our customers.”
Jeff Kimsey, Associate Vice President of Product Management for NASDAQ OMX Global Data Products
Storage
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Elastic Block Store High performance block storage device
1GB to 1TB in size
Mount as drives to instances
Feature Details High performance file
system Mount EBS as drives and format as required
Flexible size Volumes from 1GB to 1TB in size
Secure Private to your instances
Available Replicated within an Availability Zone
Backups Volumes can be snapshotted for point in time restore
Monitoring Detailed metrics captured via Cloud Watch
Storage
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Database Relational Database Service Database-as-a-Service
No need to install or manage database instances
Scalable and fault tolerant configurations
Feature Details
Platform support Create MySQL, SQL Server and Oracle RDBMS
Preconfigured Get started instantly with sensible default settings
Automated patching Keep your database platform up to date automatically
Backups Automatic backups and point in time recovery and full DB backups
Backups Volumes can be snapshotted for point in time restore
Failover Automated failover to slave hosts in event of a failure
Replication Easily create read-replicas of your data and seamlessly replicate data across availability zones
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Amazon Relational Database Service (Amazon RDS) databases stores forum threads, site content, and project configuration data. High availability Multi-AZ database deployment to handle live game metadata and user-generated content. Enterprise-grade fault tolerance for protecting customer data. By managing time-consuming database administration tasks, Amazon RDS allows SEGA to focus on business critical applications.
Database
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
DynamoDB Provisioned throughput NoSQL database
Fast, predictable performance
Fully distributed, fault tolerant architecture
Feature Details Provisioned throughput Dial up or down provisioned read/write capacity
Predictable performance
Average single digit millisecond latencies from SSD backed infrastructure
Strong consistency Be sure you are reading the most up to date values
Fault tolerant Data replicated across availability zones
Monitoring Integrated to Cloud Watch
Secure Integrates with AWS Identity and Access Management (IAM)
Elastic MapReduce Integrates with Elastic MapReduce for complex analytics on large datasets
Database
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Application Services
Amazon SQS
Processing
task/processing trigger
Processing results
Amazon SQS Reliable, highly scalable, queue service
for storing messages as they travel
between instances
Feature Details Reliable Messages stored redundantly across multiple
availability zones
Simple Simple APIs to send and receive messages
Scalable Unlimited number of messages
Secure Authentication of queues to ensure controlled access
Task A
Task B
(Auto-scaling)
Task C
2
3
1
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Feature Details Process
state Maintain application state across complex workflows in a reliable and available manner
Tracking Tracks executions and log process for audit purposes
Consistency Ensures processing tasks are executed and duplicity of events does not occur
Simple Simple Decider and Task programming model for rapid integration
Simple Workflow Reliably coordinate processing steps across
applications
Integrate AWS and non-AWS resources
Manage distributed state in complex systems
Application Services
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Cloud Search Elastic search engine based upon Amazon A9
search engine
Fully managed service with sophisticated
feature set
Scales automatically
Document
Server
Results
Search
Server
Feature Details Auto-scaling Automatically scales based upon request volumes and data
volumes
High performance
In memory operation means consistently low latency for search results
Sophisticated features
Support for faceting, stemming, synonyms, stop words and custom rank expressions
Low cost Elastic service, pay for what you use
Application Services
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
CloudFront World-wide content distribution network
Easily distribute content to end users with
low latency, high data transfer speeds,
and no commitments.
Feature Details Fast Multiple world-wide edge locations to serve content as close to
your users as possible
Integrated with other services
Works seamlessly with S3 and EC2 origin servers
Dynamic content Supports static and dynamic content from origin servers
Streaming Supports rtmp from S3 and includes support for live streaming from Adobe FMS and Microsoft Media Server
London
Paris
NY
Served from S3
/images/*
3
Served from EC2
*.php
2
Single CNAME 1
Application Services
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Deployment & Admin Elastic Beanstalk One-click deployment from Eclipse, Visual
Studio and Git
Rapid deployment of applications
All AWS resources automatically created
Feature Details Platform support Containers for Java, .net and PHP
Resource creation Creates load balancer, instances, autoscaling and monitoring automatically
Monitoring & Logs Integrated with Cloud Watch and consolidates server logs
Versioning Manage versions of applications and easily rollback deployments
Notifications Receive alerts on key events
Full resource access
Access all underlying AWS resources as necessary
Cloud Formation Automate creation of ‘stacks’ in a repeatable way
Scripting framework for AWS resource creation
Feature Details Platform support Support for AWS resources from EC2 to IAM
Resource creation Creates AWS resources behind the scenes and reports on progress
Declarative Specify stacks in JSON format and source control your environments
Customizable Drive stack creation with parameters
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Deployment & Admin
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Identity & Access Management Granular control of user rights with AWS
Automated granting of EC2 service rights
Software Developer Kits Comprehensive support of programming models for using
AWS services
Deployment & Admin
+ others Simple Email Service
Simple Notification Service ElastiCache
Elastic MapReduce CloudWatch
…
The Toolbox
The Toolbox
A toybox for dev-ops
The Toolbox
With grown up certifications
Foundation Services
Compute Storage Database Networking
AWS Global Infrastructure Regions
Availability Zones
Edge Locations Am
azo
n
Shared responsibility
Foundation Services
Compute Storage Database Networking
AWS Global Infrastructure Regions
Availability Zones
Edge Locations Am
azo
n
Shared responsibility
Sarbanes-Oxley (SOX)
ISO 27001 Certification
Payment Card Industry Data Security
Standard (PCI DSS) Level 1 Compliant
SAS70(SOC 1) Type II Audit
FISMA A&As Multiple NIST Low Approvals to Operate (ATO) NIST Moderate, GSA issued ATO FedRAMP
DIACAP MAC III Sensitive IATO Customers have deployed various compliant applications such as HIPAA (healthcare)
Foundation Services
Compute Storage Database Networking
AWS Global Infrastructure Regions
Availability Zones
Edge Locations
Client-side Data Encryption & Data Integrity Authentication
Server-side Encryption (File System and/or Data)
Network Traffic Protection (Encryption/Integrity/Identity)
Platform, Applications, Identity & Access Management
Operating System, Network & Firewall Configuration
Customer Data
Am
azo
n
You
Shared responsibility