growing x20 without spending an extra penny on hosting
DESCRIPTION
A year ago we started a big change in FTBpro. We completely changed the visual design, moved to a single page architecture and started exploring new ways to minimise load on our servers - both when serving our actual website, and mobile API responses.We'll focus on how scaling considerations are now an integral part of our architecture, which enabled us to serve 20x more traffic than we did 1 year ago, with the same setup and with no additional costs.TRANSCRIPT
Growing X20 without spending an extra penny on hosting
Dor Kalev, CTO @ FTBpro
Largest fan-generated media platform in global football
130m pageviews / month
a Football fan #musthave1M+ mobile installs
Content is king
Single Page Application
Phantom.js renders pages on the server-side for SEO, crawlers and mobile web performance
Load Balancer
Memcached
Redis
Web 1 Web 2 Web 3
MySQL MongoDB ElasticSearch
Amazon Web Services
NginX +
Ruby on Rails
PhantomJS
Web 4
How do we scale post pages?
Each Render Server can return 1 client request at a time
1:1Client Server
Render
Linear correlation between clients and servers
1:1Client Server
Render
Client ServerRender
A cache server can handle over x10 more traffic
10:1:1
Cache Server Render
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Cache server still consumes CPU and with more traffic, more servers are needed
20:2:1
Cache
Server Render
Cache
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Accelerator like NGINX+MemCached or Varnish can be x100 more efficient than
plain App Server Cache100:1:1
Server Render
Accelerator
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
https://www.igvita.com/2008/02/11/nginx-and-memcached-a-400-boost/https://www.varnish-cache.org
200:2:1
Server Render
Accelerator
Accelerator
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Accelerator also runs on limited hardware and needs more CPU in order to scale
Enter CDN∞:1
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
CDN Server Render
ClientClient
My Tiny
Server
PoPCDN
SERVERCDN
SERVER
CDN SERVER
PoPCDN
SERVERCDN
SERVER
CDN SERVER
PoPCDN
SERVERCDN
SERVER
CDN SERVER
Client
Client Client Client Client Client
Client
What’s a CDN?
What’s a CDN?
• Content Delivery Network
• A set of autonomic distributed cache servers
• Serve cache from a location geographically close to the client
• e.g: Akamai, Cloudflare, Fastly, Highwinds, Edgecast, Cloudfront
Enter Trouble∞:1
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
Client
CDN Server Render
Fixing minor Typo? Need to Refresh Cache??
TTL vs. ON CHANGE
Cache Policy
Purge by URL can take over an hour on classic CDNs
Z Z Z
Purge by TTL - isn’t realtime & causes redundant server hits
CPU
LO
AD
0
7.5
15
22.5
30
TIME
1 2 3 4 5 (TTL) 6 7 8 9 (TTL) 10 11 12
multiply by all articlesC
PU L
OAD
0
7.5
15
22.5
30
TIME
1 2 3 4 5 (TTL) 6 7 8 9 (TTL) 10 11 12
TTL = Constant waste of CPUC
PU L
OAD
0
7.5
15
22.5
30
TIME
1 2 3 4 5 6 7 8 9 10 11 12
Better Solution?
abuse new CDN technologiesSuper Fast Purge + Smart API!
!!!!!
e.g. Fastly, EdgeCast
cache forever +purge on demand
0
7.5
15
22.5
30
1 (release) 2 3 4 5 6 7 8 9 (purge) 10 11 12
fast purge at ~10-45 seconds
Advantages
• ∞ scaling
• cheap
• fast
• keep the stack simple
Caveats
• Purging everything can lead to downtime, e.g. when deploying
• User Specific content can’t be cached with the entire page
• Downtime on massive Push Notifications
Massive Push Notification + cold CDN Cache = traffic goes to app servers
CPU
LO
AD
0
250
500
750
1000
TIME
1 (PN) 2 3 4 5 6 7 8 9 10 11 12
ClientClient
My Tiny
Server
PoPCDN
SERVERCDN
SERVER
CDN SERVER
PoPCDN
SERVERCDN
SERVER
CDN SERVER
PoPCDN
SERVERCDN
SERVER
CDN SERVER
Client
Client Client Client Client Client
Client
Render (or cache & serve) X number of calls in parallel = self-DDoS
ClientClient
CACHE
PoPCDN
SERVERCDN
SERVER
CDN SERVER
PoPCDN
SERVERCDN
SERVER
CDN SERVER
PoPCDN
SERVERCDN
SERVER
CDN SERVER
Client
Client Client Client Client Client
Client
Solution: Another local cache layer & Preload CDN (few CDNs have this feature, e.g. EdgeCast)
My Tiny
Server
CPU vs TRAFFIC
• 13:53 preloaded post to CDN • 13:55 PN sent • 13:57+ traffic stops @ CDN servers, no CPU throughput • 14:00 …
14:00 Laugh a maniacal laugh of satisfaction
Summary• CDN allows massive scale
• Smart CDN usage allows even more scale
• Scale + fresh content + low hosting prices is a do-able mission
!
Questions?
Read More…• Growing X20 without spending an extra penny on
hosting (The original blog post)http://tech.ftbpro.com/post/78969626647/growing-x20-without-spending-an-extra-penny-on-hosting
• Single Page Applications Done Righthttp://tech.ftbpro.com/post/84720180206/single-page-applications-done-right
• Count von Count - Real time counting DBhttp://tech.ftbpro.com/post/73212710968/count-von-count-a-real-time-counting-database
• The Dark Side of Single Page Applicationshttp://tech.ftbpro.com/post/66182501420/the-dark-side-of-single-page-applications-rails
OSS: http://github.com/ftbproBLOG: http://tech.ftbpro.com