aws at argylesocial
TRANSCRIPT
![Page 1: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/1.jpg)
How AWS Drives Argyle Social
![Page 2: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/2.jpg)
A bit about Argyle Social
• What does Argyle do?
– Social Media management, engagement, & Analytics
• We are a small & very efficient dev team
– Adam (general), Mike (platform), & Josh (design)
![Page 3: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/3.jpg)
Social
AWS is Startup Friendly
RDS
SimpleDB
Feb ‘10
‘The Server’
MySql
Dec ‘10
App ServerSimple
DBMySql SQS
SimpleDB
May ‘10
‘The Server’
MySql
SQS
![Page 4: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/4.jpg)
Servers Galore
RDS
Jun ‘11
Jobs
SimpleDB
SQS
SocialSocialSocialStream
MySqlMySql
App ServerURL
Redir
S3 Support
![Page 5: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/5.jpg)
Our current setup
• EC2 – 9 Servers
• SQS – 5 Queues
• SimpleDB – 1 Domain
• RDS – 1 DB; 5MM records
• Support & S3 – Just starting out
• Let’s dig into SQS, SimpleDB, & Support
![Page 6: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/6.jpg)
SQS – Simple Queue Service
• Used for processing:– Twitter Streams
– Email queue
– Filtering for significant items
– Log Processing for Goals
• Very easy to get started:
![Page 7: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/7.jpg)
How it works
Process Everything
Queue Everything
Process MessageProcess Message
Process MessageProcess Message
Queue
![Page 8: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/8.jpg)
Argyle SQS Usage
AWS Service Land
Interweb
Streams RDS Database
Twitter Site Streams
Queue
Twitter Processor
Twitter Processor
Twitter Processor
Twitter Stream Reader
Twitter Stream Reader
Twitter Stream Reader
Argyle Web Interface
Users
![Page 9: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/9.jpg)
SQS – What we learned
• Not super fast– Lag between pushing and seeing an item on pull
• Visibility timeout is key– Make sure you have time to finish and delete– ‘at least once delivery’ you may get something twice
• Not guaranteed FIFO• Default size
– Restrictive Sizing can lead to errors– 1kB to 64kB
• Getting Queue Url– Create it and store it, don’t look it up
![Page 10: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/10.jpg)
SQS - Conclusions
• Very easy to get bullet-proof queues
– No administration costs
• Expect some limitations that rolling or running your own won’t incur
– MSMQ, Spread, etc
![Page 11: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/11.jpg)
Simple DB
• Easy storage & retrieval of Key/Value pairs
• Eventual consistent read vs. consistent read
• Grows with your data
• NoSQL N00bs: take your time
– This isn’t relational, which is obvious, but may take some time to really wrap your head around
– No JOIN or GROUP BY, etc
![Page 12: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/12.jpg)
Argyle SimpleDB Usage
Redirector
Memcached
SimpleDB
Redirecting a URL
ar.gy/123 ar.gy/12
AWS Service Land
Interweb
Cached URL Non-cached URL
Click log
Redirector
Memcached
SimpleDB
ar.gy/ftw
Shortening a URL
![Page 13: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/13.jpg)
SDB – Scratch Pad
*This makes your life easier, especially when starting out or debugging
![Page 14: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/14.jpg)
SDB – Lessons Learned
• Be aware of the limits:
Source: http://en.wikipedia.org/wiki/Amazon_SimpleDB#Limitations
![Page 15: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/15.jpg)
SDB – Lessons Learned
• We’ve had some availability issues– Sometimes no response
– Sometimes no write capabilities
• Storage of big items is possible, but must be linked out
• Can separate domains to increase performance
![Page 16: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/16.jpg)
SimpleDB - Conclusions
• Great, easy storage
• Again, reduced maintenance costs
• Have to change your mindset from RDBMS
![Page 17: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/17.jpg)
AWS Support
• We’re new to it
• We don’t expect much
• It can be helpful toget nagging questionsresolved
• This is your company on the line, right?
![Page 18: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/18.jpg)
AWS Support
• We ‘lost’ a server; 4.5 hours after submitted:
• Clarification on “underlying host issue” pls; 40 minutes to reply
![Page 19: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/19.jpg)
Support Conclusions
• Well worth the security
• Rumor has it that upgrading to higher levels isn’t worth it – any experiences out there?
– (Update: we got good feedback around higher levels of service from Adzerk)
• You still have to design for failure
![Page 20: AWS at ArgyleSocial](https://reader033.vdocuments.us/reader033/viewer/2022060201/559b29781a28ab5f2c8b4608/html5/thumbnails/20.jpg)
Q & A
• About Argyle
• About your own AWS setup