amazon web services: building highly scalable web applications institutional web management workshop...
TRANSCRIPT
![Page 1: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/1.jpg)
Amazon Web Services:Building Highly Scalable Web Applications
Institutional Web Management WorkshopJuly 2007
Jeff Barr
Senior Web Services Evangelist
![Page 2: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/2.jpg)
Today
Introduction
Amazon Web Services overview
Amazon EC2
Amazon S3
Amazon SQS
Application Architecture
Questions
![Page 3: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/3.jpg)
Who am I?
Software development background
5 Years with Amazon:
Senior Developer
Web Services Evangelist
Senior Web Services Evangelist
Veteran of several startups
Microsoft .Net Team
Visix Software co-founder
![Page 4: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/4.jpg)
Amazon?
Consumers (Buy)
Associates, Sellers & Merchants (Sell)
Developers (Innovate)
![Page 5: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/5.jpg)
Issues Facing Developers
70% of Web Development Effort is “Muck”:
Data Centers
Bandwidth / Power / Cooling
Operations
Staffing
Scaling is Difficult and Expensive:
Large Up-Front Investment
Invest Ahead of Demand
Load is Unpredictable
![Page 6: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/6.jpg)
Dream or Nightmare?
Slashdot/Digg/TechCrunch Effect
Rapid, unexpected customer demand/growth
![Page 7: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/7.jpg)
Seasonal Spikes
![Page 8: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/8.jpg)
More Seasonal Spikes
“Every year, we take the busiest minute of the busiest hour of the busiest day and build capacity on that, we built our systems to (handle that load) and we went above and beyond that.” *
-- Scott Gulbransen
Intuit Spokesman
* http://news.com.com/2100-1038_3-6177341.html
![Page 9: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/9.jpg)
Solution: Web-Scale Computing
Scale capacity on demand
Turn fixed costs into variable costs
Always available
Rock-solid reliability
Simple APIs and conceptual models
Cost-effective
Reduced time to market
Focus on product & core competencies
![Page 10: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/10.jpg)
10
Amazon Web Services Are...
A set of APIs and business models which give developer-level access to Amazon’s infrastructure and content:
Data As A ServiceAmazon E-Commerce Service
Amazon Historical Pricing
Search As A ServiceAlexa Web Information Service
Alexa Top Sites
Alexa Site Thumbnail
Alexa Web Search Platform
Infrastructure As A ServiceAmazon Simple Queue Service
Amazon Simple Storage Service
Amazon Elastic Compute Cloud
People As A ServiceAmazon Mechanical Turk
![Page 11: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/11.jpg)
The Commercial Side
Standard licensing terms
Commercially usable
Aggressive pricing
Monthly credit card billing
Self-serve model:
Sign up as developer
Choose services
Agree to service licenses
Enter payment info
Start coding
![Page 12: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/12.jpg)
Web Services Billing
![Page 13: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/13.jpg)
Infrastructure Services
Compute
Store Message
Elastic ComputeCloud
Simple Storage Service
Simple QueueService
![Page 14: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/14.jpg)
Amazon Simple Storage Service
S3
![Page 15: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/15.jpg)
Amazon Simple Storage ServiceAmazon Simple Storage Service
$.15 per GB per month
storage
• Object-Based Storage• 1 B – 5 GB / object• Fast, Reliable, Scalable• Redundant, Dispersed• 99.99% Availability Goal• Private or Public• Per-object URLs & ACLs• BitTorrent Support $.10 - $.18 per
GB data transfer
$.01 for 1000 to 10000 requests
![Page 16: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/16.jpg)
Amazon S3 Concepts
Objects:Opaque data to be stored (1 byte … 5 Gigabytes)Authentication and access controls
Buckets:Object container – any number of objects100 buckets per account / buckets are “owned”
Keys:Unique object identifier within bucketUp to 1024 bytes longFlat object storage model
Standards-Based Interfaces:REST and SOAPURL-Addressability – every object has a URL
![Page 17: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/17.jpg)
S3 SOAP/Query APIService:
ListAllMyBuckets
Buckets:CreateBucketDeleteBucketListBucketGetBucketAccessControlPolicySetBucketAccessControlPolicyGetBucketLoggingStatusSetBucketLoggingStatus
Objects:PutObjectPutObjectInlineGetObjectGetObjectExtendedDeleteObjectGetObjectAccessControlPolicySetObjectAccessControlPolicy
![Page 18: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/18.jpg)
![Page 19: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/19.jpg)
![Page 20: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/20.jpg)
Amazon Simple Queue Service
SQS
![Page 21: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/21.jpg)
Amazon Simple Queue ServiceAmazon Simple Queue Service
$.10 per 1000 messages
• Scalable Queuing• Elastic Capacity• Reliable, Simple, Secure
Inter-process messaging, data buffering, architecture component
$.10 - $.18 per GB data transfer
![Page 22: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/22.jpg)
Amazon SQS Concepts
Queues:Named message containerPersistent
Messages:Up to 256KB of data per messagePeek / Lock access model
Scalable:Unlimited number of queues per accountUnlimited number of messages per queue
![Page 23: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/23.jpg)
Amazon SQS At Work
![Page 24: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/24.jpg)
SQS SOAP/Query APIQueues:
ListQueues DeleteQueueSetVisibilityTimeoutGetVisibilityTimeout
Messages: SendMessage ReceiveMessage DeleteMessage PeekMessage
Security:AddGrantListGrantsRemoveGrant
![Page 25: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/25.jpg)
Amazon Elastic Compute Cloud
EC2
![Page 26: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/26.jpg)
Amazon Elastic Compute CloudAmazon Elastic Compute Cloud
$.10 per server hour
• Virtual Compute Cloud• Elastic Capacity• 1.7 GHz x86• 1.7 GB RAM• 160 GB Disk• 250 MB/Second Network• Network Security Model
Time or Traffic-based Scaling, Load testing, Simulation and Analysis, Rendering, Software as a Service Platform, Hosting
$.10 - $.18 per GB data transfer
![Page 27: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/27.jpg)
Amazon EC2 Concepts
Amazon Machine Image (AMI):Bootable root diskPre-defined or user-builtCatalog of user-built AMIsOS: Fedora, Centos, Gentoo, Debian, Ubuntu, Windows ServerApp Stack: LAMP, mpiBLAST, Hadoop
Instance:Running copy of an AMILaunch in less than 2 minutesStart/stop programmatically
Network Security Model:Explicit access controlSecurity groups
Inter-service bandwidth is free
![Page 28: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/28.jpg)
Amazon EC2 At Work
StartupsCruxy – Media transcodingGigaVox Media – Podcast Management
Fortune 500 clients:High-Impact, S hort-Term ProjectsDevelopment Host
Science / Research:Hadoop / MapReducempiBLAST
Load-Management and Load Balancing Tools:Pound WeogeoRightscale
![Page 29: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/29.jpg)
EC2 SOAP/Query API
Images:RegisterImageDescribeImagesDeregisterImage
Instances:RunInstancesDescribeInstancesTerminateInstancesGetConsoleOutputRebootInstances
Keypairs:CreateKeyPairDescribeKeyPairsDeleteKeyPair
Image Attributes:
ModifyImageAttribute
DescribeImageAttribute
ResetImageAttribute
Security Groups:
CreateSecurityGroup
DescribeSecurityGroups
DeleteSecurityGroup
AuthorizeSecurityGroupIngress
RevokeSecurityGroupIngress
![Page 30: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/30.jpg)
![Page 31: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/31.jpg)
Sample Web-Scale Architecture
GigaVox Media
![Page 32: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/32.jpg)
Web-Scale Architecture
![Page 33: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/33.jpg)
GigaVox Economics
Implemented Amazon S3, Amazon EC2 and Amazon SQS in November 2006
Created an infinitely scalable infrastructure for less than $100 - building the same infrastructure themselves would have cost thousands of dollars
Reduced staffing requirements - far less responsibility for 24x7 operations
![Page 34: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/34.jpg)
Q & A
![Page 35: Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist](https://reader035.vdocuments.us/reader035/viewer/2022062619/55160884550346a2308b4f75/html5/thumbnails/35.jpg)
Thank You!
PointersPortalhttp://aws.amazon.com
Bloghttp://aws.typepad.com
EC2http://aws.amazon.com/ec2
S3http://aws.amazon.com/s3
Resource Centerhttp://aws.amazon.com/resources
Forumshttp://aws.amazon.com/forums
Jeff BarrSenior Web Services Evangelist