growing x20 without spending an extra penny on hosting

Growing X20 without spending an extra penny on hosting

Dor Kalev, CTO @ FTBpro

Largest fan-generated media platform in global football

130m pageviews / month

a Football fan #musthave1M+ mobile installs

Content is king

Single Page Application

Phantom.js renders pages on the server-side for SEO, crawlers and mobile web performance

Load Balancer

Memcached

Redis

Web 1 Web 2 Web 3

MySQL MongoDB ElasticSearch

Amazon Web Services

NginX +

Ruby on Rails

PhantomJS

Web 4

How do we scale post pages?

Each Render Server can return 1 client request at a time

1:1Client Server

Render

Linear correlation between clients and servers

1:1Client Server

Render

Client ServerRender

A cache server can handle over x10 more traffic

10:1:1

Cache Server Render

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Cache server still consumes CPU and with more traffic, more servers are needed

20:2:1

Cache

Server Render

Cache

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Accelerator like NGINX+MemCached or Varnish can be x100 more efficient than

plain App Server Cache100:1:1

Server Render

Accelerator

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

https://www.igvita.com/2008/02/11/nginx-and-memcached-a-400-boost/https://www.varnish-cache.org

200:2:1

Server Render

Accelerator

Accelerator

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Accelerator also runs on limited hardware and needs more CPU in order to scale

Enter CDN∞:1

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

CDN Server Render

ClientClient

My Tiny

Server

PoPCDN

SERVERCDN

SERVER

CDN SERVER

PoPCDN

SERVERCDN

SERVER

CDN SERVER

PoPCDN

SERVERCDN

SERVER

CDN SERVER

Client

Client Client Client Client Client

Client

What’s a CDN?

What’s a CDN?

• Content Delivery Network

• A set of autonomic distributed cache servers

• Serve cache from a location geographically close to the client

• e.g: Akamai, Cloudflare, Fastly, Highwinds, Edgecast, Cloudfront

Enter Trouble∞:1

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

Client

CDN Server Render

Fixing minor Typo? Need to Refresh Cache??

TTL vs. ON CHANGE

Cache Policy

Purge by URL can take over an hour on classic CDNs

Z Z Z

Purge by TTL - isn’t realtime & causes redundant server hits

CPU

LO

AD

0

7.5

15

22.5

30

TIME

1 2 3 4 5 (TTL) 6 7 8 9 (TTL) 10 11 12

multiply by all articlesC

PU L

OAD

0

7.5

15

22.5

30

TIME

1 2 3 4 5 (TTL) 6 7 8 9 (TTL) 10 11 12

TTL = Constant waste of CPUC

PU L

OAD

0

7.5

15

22.5

30

TIME

1 2 3 4 5 6 7 8 9 10 11 12

Better Solution?

abuse new CDN technologiesSuper Fast Purge + Smart API!

!!!!!

e.g. Fastly, EdgeCast

cache forever +purge on demand

0

7.5

15

22.5

30

1 (release) 2 3 4 5 6 7 8 9 (purge) 10 11 12

fast purge at ~10-45 seconds

Advantages

• ∞ scaling

• cheap

• fast

• keep the stack simple

Caveats

• Purging everything can lead to downtime, e.g. when deploying

• User Specific content can’t be cached with the entire page

• Downtime on massive Push Notifications

Massive Push Notification + cold CDN Cache = traffic goes to app servers

CPU

LO

AD

0

250

500

750

1000

TIME

1 (PN) 2 3 4 5 6 7 8 9 10 11 12

ClientClient

My Tiny

Server

PoPCDN

SERVERCDN

SERVER

CDN SERVER

PoPCDN

SERVERCDN

SERVER

CDN SERVER

PoPCDN

SERVERCDN

SERVER

CDN SERVER

Client

Client Client Client Client Client

Client

Render (or cache & serve) X number of calls in parallel = self-DDoS

ClientClient

CACHE

PoPCDN

SERVERCDN

SERVER

CDN SERVER

PoPCDN

SERVERCDN

SERVER

CDN SERVER

PoPCDN

SERVERCDN

SERVER

CDN SERVER

Client

Client Client Client Client Client

Client

Solution: Another local cache layer & Preload CDN (few CDNs have this feature, e.g. EdgeCast)

My Tiny

Server

CPU vs TRAFFIC

• 13:53 preloaded post to CDN • 13:55 PN sent • 13:57+ traffic stops @ CDN servers, no CPU throughput • 14:00 …

14:00 Laugh a maniacal laugh of satisfaction

Summary• CDN allows massive scale

• Smart CDN usage allows even more scale

• Scale + fresh content + low hosting prices is a do-able mission

!

Questions?

Read More…• Growing X20 without spending an extra penny on

hosting (The original blog post)http://tech.ftbpro.com/post/78969626647/growing-x20-without-spending-an-extra-penny-on-hosting

• Single Page Applications Done Righthttp://tech.ftbpro.com/post/84720180206/single-page-applications-done-right

• Count von Count - Real time counting DBhttp://tech.ftbpro.com/post/73212710968/count-von-count-a-real-time-counting-database

• The Dark Side of Single Page Applicationshttp://tech.ftbpro.com/post/66182501420/the-dark-side-of-single-page-applications-rails

OSS: http://github.com/ftbproBLOG: http://tech.ftbpro.com