scaling for web - architecting hootsuite for 5+ million users
TRANSCRIPT
Scaling for WebArchitecting HootSuite for 5 Million+ Users
About Us
Beier CaiChief [email protected]@beiercai
Bill MonkmanLead Developer, [email protected]@bmonkman
Hoo’s HootSuite
Some Numbers
5+ million Members
10+ million Social Networks
25+ million Streams
350+ million Messages Scheduled
1.3+ Billion Messages Sent
2+ Billion Page Views per month
2,000 Web Requests/sec
Technologies
PHP
MySQL
Technologies
PHP
Memcached
MySQL
Technologies
PHP
Gearman
Memcached
MySQL
Technologies
PHP
Gearman
Memcached
MySQL MongoDB
Technologies
PHP
Gearman ZeroMQ
Memcached
MySQL MongoDB
Technology Stack
LAMP (Linux, Apache, MySQL, PHP)
Scala
Akka
Python
Clojure
MongoDB
Redis
Memcached
Gearman
ZeroMQ
RabbitMQ/AMQP
jQuery/Backbone
EC2 Cloud / CloudStack
Jenkins / CruiseControl
CloudFront CDN
HAProxy/Nginx/Varnish
Zend
Statsd/Graphite
LogStash
ElasticSearch/Lucene
Nagios/Munin/Sensu
ZooKeeper
Optimization
• Push processing up to the front-end, make your users’ computers into a distributed processing network. e.g. stream rendering
• Cache as much as possible, invalidate only when necessary.
• Sometimes features have to take a back-seat to performance/scalability.
• Look at profiler data.
• Watch query execution time, optimize when necessary with indexes, partitions, de-normalization, etc.
• Know what your weak spots are and keep a close eye on them.
Optimization
Optimization Case Study
Optimization Case Study
Scaling Dev Team
• Passion
• Commitment
• Dedication
• Responsibility
• General Awesomeness
Besides technical competency, we also value
Scaling Dev Team
Passion
• Commitment
• Dedication
• Responsibility
• General Awesomeness
Besides technical competency, we also value
Scaling Dev Team
Team Work
Entrepreneurial
• Communication
• Respect
• Trust
Coding Practice
• Clean and Readable
• Maintainable
• Reusable
• Scalable
• Commented
Have a well defined coding standard within your dev team, and follow it!
Monitoring, Profiling & Logging
Monitoring, Profiling & Logging
Case Study
Case Study
