SPOT301: November 12, 2014 | Las Vegas, NV
James Hamilton, AWS VP & Distinguished Engineer
“5X the cloud capacity in use than the
aggregate total of the other 14 providers”
Every day, AWS adds enough new server capacity to support all of Amazon’s global
infrastructure when it was a $7B annual revenue enterprise
57%
8%
18%
13% 4% Servers
NetworkingEquipment
Power Distribution &Cooling
Power
Other Infrastructure
Monthly Costs
3yr server & 10 yr infrastructure amortization
AZ
AZ
AZ AZ AZ
Transit
Transit
• Asynchronous replication between distant data centers
• Committing to an SSD order 1 to 2ms
• But, LA to New York is 74ms round trip
• You can’t wait 74ms to commit a transaction
• On failure, difficult & high skill decision: • Fail-over & lose transactions, or
• Or don’t fail-over & lose availability
• Difficult choice
• Use AZs for no-admin failover • Sync works when <2ms
• Can be combined with regional replication for very high availability
74ms
AZ
AZ
AZ AZ AZ
Transit
Transit
26%
40%
25%
30%
35%
40%
Multi-AZ RDS % of all RDS
Synchronous Replication
AZ1 AZ2
SQL
Txns
Caching
Store Store
Transform: Write to
Read Format
Optimized
Store Store
AZ 1 AZ 2
Store Store
AZ 3
Transform: Write to
Read Format
Optimized
Transform: Write to
Read Format
Optimized
AZ 1 AZ 2 AZ 3
Full 10Gbps Links
Ingestion,
Backup, &
Restore
Customer VPC
Internal
VPC
JDBC/ODBC
EBS
CaptureSubmissions
Process in Realtime
Store inRedshift
ClientsSubmitting
Data
AWS Metering:
• Tens of millions records/sec
• Multiple TB per hour
• 100,000s of internal sources
• Scales, low-cost, auditable,
with real time alerting
AWS Kinesis:
• Producers call put
• Sequence # returned
• Distributed over shards
• Scales per shard at 1
MB/s & 1000 TPS
Amazon Web Services
AZ AZ AZ
Durable, highly consistent storage replicates dataacross three data centers (availability zones)
Aggregate andarchive to S3
Millions ofsources producing100s of terabytes
per hour
FrontEnd
AuthenticationAuthorization
Ordered streamof events supportsmultiple readers
Real-timedashboardsand alarms
Machine learningalgorithms or
sliding windowanalytics
Aggregate analysisin Hadoop or adata warehouse
Inexpensive: $0.028 per million puts
2008 2009 2010 2011 2012 2013 2014
On pace for over 400 new
services or major features in 2014
280
159
82 61
48 24
447
Thank You!