spa2015: hooman beheshti – the future of cdns

Evolution  of  the  CDN

Hooman  Beheshti,  VP  Technology

SPA2015

CDN

• Globally  distributed  network  of  servers

• Built  primarily  with  caching  engines  (squid,  nginx,  varnish,  etc)

• Footprints  vary

CDN  benefits

• Performance• Latency  decrease  due  to  footprint

• Origin  offload

• Much  much  more…

Deployment

• Originally  simple:• Serve  root  web  page  (HTML)  direct  from  origin• Point  static  assets  on  the  page  to  a  domain  owned  by  the  CDN

• Everything  controlled  through  DNS

Types  of  content

CacheableUncacheable

Static

Dynamic

Cache-Control: max-age=31536000

Dynamic  Content

Transport  and  delivery  optimization

Transport  and  delivery  optimization

Application  delivery  platform

Things  have  started  changing…

Applications  have  became  smarter

Caching  has  gotten  more  complicated

CacheableUncacheable

Static

Dynamic

CacheableUncacheable

StaticDynamic

Static  =  Cacheable

All  Else  =  Dynamic  =  Uncacheable

Static  =  Cacheable

All  Else  =  Dynamic  =  Uncacheable

Types  of  content

CacheableUncacheableStatic

Dynamic

EventDriven

Static  for  unpredictable  periods  of  time

Event  Driven  Content:

Three  types  of  content

• Static• Infrequent  changes,  can  cache  for  a  long  time• Cache-­‐control  headers  are  often  good  enough• Examples:  images,  javascript,  css,  etc

• Dynamic• Can’t  cache  at  all  – must  go  to  origin• DSA  and  transport  optimization  is  key  for  performance• Examples:  logins,  credit  card  transactions,  Ajax,  etc

• Event-­‐driven:• Static,  but  unpredictably• Cache-­‐control  headers  are  not  good  enough  on  their  own• Examples:  news  stories,  wiki  pages,  user  comments,  sports  scores,  etc

Three  types  of  content

• Static• Infrequent  changes,  can  cache  for  a  long  time• Cache-­‐control  headers  are  often  good  enough• Examples:  images,  javascript,  css,  etc

• Dynamic• Can’t  cache  at  all  – must  go  to  origin• DSA  and  transport  optimization  is  key  for  performance• Examples:  logins,  credit  card  transactions,  Ajax,  etc

• Event-­‐driven:• Static,  but  unpredictably• Cache-­‐control  headers  are  not  good  enough  on  their  own• Examples:  news  stories,  wiki  pages,  user  comments,  sports  scores,  etc

Three  types  of  content

• Static• Infrequent  changes,  can  cache  for  a  long  time• Cache-­‐control  headers  are  often  good  enough• Examples:  images,  javascript,  css,  etc

• Dynamic• Can’t  cache  at  all  – must  go  to  origin• DSA  and  transport  optimization  is  key  for  performance• Examples:  logins,  credit  card  transactions,  Ajax,  etc

• Event-­‐driven:• Static,  but  unpredictably• Cache-­‐control  headers  are  not  good  enough  on  their  own• Examples:  news  stories,  wiki  pages,  user  comments,  sports  scores,  etc

Long-­‐tail  Content(cacheable  but  seldom  fetched)

Usage  patterns  have  changed

CDN  problems

• Services  were  black  boxes• Lots  of  professional  services• Caching  efficiency  has  decreased• Not  a  lot  of  visibility• No  real-­‐time  feedback• No  real-­‐time  interfaces

• APIs• Not  enough  real-­‐time  control

• Removing  content  from  the  CDN• Configuration  changes

CDNs  have  to  evolve…

Caching

Caching  event-­‐driven  content

• Caching  predictably  static  content  is  relatively  straight  forward• Headers• Configuration

• Caching  unpredictably  static  content  is  really  difficult• We  don’t  know  lifetime  in  advance• Trigger-­‐based  invalidation

Cache-Control: max-age=600

Uncaching

Purging

Purging  content  from  a  CDN

• Event-­‐driven  content  can  be  cached  if  the  CDN  allows  instantaneous  programmatic  purging• Cache  normally• Purge  when  change  trigger  happens• Rinse  and  repeat!

• Slow  purge  times  unacceptable  in  this  case

• We  need  instant purging• Deterministic  and  predictable

Example:CMS  +  Purge

WordPress:  Then

CDN  Node

WordPress:  Then

CDN  Node

WordPress:  Then

CDN  Node

WordPress:  Then

CDN  Node

WordPress:  Then

CDN  Node

Cache

WordPress:  Now

CDN  Node

WordPress:  Now

CDN  Node

HTTP/1.1 200 OKContent-Type: text/htmlContent-Length: 55,666Cache-Control: Totally Long Time!

WordPress:  Now

CDN  Node

HTTP/1.1 200 OKContent-Type: text/htmlContent-Length: 55,666Cache-Control: Totally Long Time!

WordPress:  Now

CDN  Node

WordPress:  Now

CDN  Node

WordPress:  Now

CDN  NodePURGE

WordPress:  Now

CDN  NodePURGE

WordPress:  Now

CDN  Node

(Has  to  be  instantaneous!)

PURGE

WordPress:  Now

CDN  Node

HTTP/1.1 200 OKContent-Type: text/htmlContent-Length: 55,666Cache-Control: Totally Long Time!

WordPress:  Now

CDN  Node

Long  tail  content  and  

cache  hit  ratio

Origin

Origin

Cache

TCP

Origin

Cache

HTTP

Origin

Cache

Origin

Cache

HTTP

Origin

Cache

HTTP

Origin

Cache

HTTP

Origin

Cache

HOT COLD

Origin

Cache

cache  “hit”

Popular Medium  Tail  (1hr) Long  tail    (6hr)

Popular Medium  Tail  (1hr) Long  tail    (6hr)

Connect  (median)Popular 14msec1hr  Tail 15msec

6hr Tail 16msec

Popular Medium  Tail  (1hr) Long  tail    (6hr)

Connect  (median)Popular 14msec1hr  Tail 15msec

6hr Tail 16msec

Popular Medium  Tail  (1hr) Long  tail    (6hr)

Connect  (median)Popular 14msec1hr  Tail 15msec

6hr Tail 16msec

Popular Medium  Tail  (1hr) Long  tail    (6hr)

Connect  (median)Popular 14msec1hr  Tail 15msec

6hr Tail 16msec

Popular Medium  Tail  (1hr) Long  tail    (6hr)

Connect  (median) Wait (median)Popular 14msec 19msec1hr  Tail 15msec 26msec

6hr Tail 16msec 32msec

Popular Medium  Tail  (1hr) Long  tail    (6hr)

Connect  (median) Wait (median)Popular 14msec 19msec1hr  Tail 15msec 26msec

6hr Tail 16msec 32msec 6,400+  measurements

77,000+  measurements

38,000+  measurements

Popular Medium  Tail  (1hr) Long  tail    (6hr)

Isn’t  this  better?

Object  storage

Caching

• Cache  more

• Serve  more  from  the  edge

• Control  caching  and  uncaching,  instantaneously  

Control

Programmability

• Granular  invalidation  API

• Configuration  API

• Logic  at  the  edge

• All  configuration  interaction  must  be  instantaneous

Control  at  the  edge

• Moving  application  logic  to  the  edge

• Example:  VCL  (Varnish  Configuration  Language)• Script-­‐like  configuration  for  functionality  at  the  edge

Control  at  the  edge

• Moving  application  logic  to  the  edge

• Example:  VCL  (Varnish  Configuration  Language)• Script-­‐like  configuration  for  functionality  at  the  edge

• Not  exclusive  to  varnish• Any  mechanism  offered  by  the  CDN  to  allow  logic  to  be  executed  at  the  edge  is  good

Logic  at  the  edge

• Generate  content  at  the  edge• HTTP  header  manipulation• Origin  selection• Caching  rules• Geo-­‐IP  rules• Forcing  SSL• Serving  stale  content  • Etc,  etc,  etc

Applications  without  origins?

Control

• Programmatic  interfaces

• Real-­‐time  interaction  with  the  CDN

• Running  logic  at  the  edge

Visibility

Real-­‐time  analytics

• Statistics  API• Network  stats• HTTP  stats  (status  codes,  etc)• Caching  stats  (hits,  misses,  errors,  etc)• Everything  has  to  be  real-­‐time• Historic  data  is  also  a  must

Logging

• Daily  or  hourly  logs  are  not  good  enough

• Logs  in  real-­‐time

• Log  streaming• To  any  logging  destination  endpoint• Syslog,  S3,  FTP,  etc.

Visibility

• We  need  to  see  what’s  going  on

• Real-­‐time  stats• Stats  API

• Real-­‐time  logs

Example:Beacon  Termination  at  the  Edge

Then

CDN  Node

Origin

Log  Analysis

http://collector.site.com/beacon.img?a=1&b=2&c=3

Then

CDN  Node

Origin

Log  Analysis

http://collector.site.com/beacon.img?a=1&b=2&c=3

Then

CDN  Node

Origin

Log  Analysis

HTTP/1.1 200 OKExpires: Wed, 19 Apr 2000 11:43:00 GMTCache-Control: no-cache, no-storeLast-Modified: Wed, 21 Jan 2004 19:51:30 GMTContent-Type: image/gifDate: Fri, 20 Jun 2014 12:22:20 GMTServer: ApacheContent-Length: 35

http://collector.site.com/beacon.img?a=1&b=2&c=3

Then

CDN  Node

Origin

Log  Analysis

HTTP/1.1 200 OKExpires: Wed, 19 Apr 2000 11:43:00 GMTCache-Control: no-cache, no-storeLast-Modified: Wed, 21 Jan 2004 19:51:30 GMTContent-Type: image/gifDate: Fri, 20 Jun 2014 12:22:20 GMTServer: ApacheContent-Length: 35

http://collector.site.com/beacon.img?a=1&b=2&c=3

Then

CDN  Node

Origin

Log  Analysis

HTTP/1.1 200 OKExpires: Wed, 19 Apr 2000 11:43:00 GMTCache-Control: no-cache, no-storeLast-Modified: Wed, 21 Jan 2004 19:51:30 GMTContent-Type: image/gifDate: Fri, 20 Jun 2014 12:22:20 GMTServer: ApacheContent-Length: 35

http://collector.site.com/beacon.img?a=1&b=2&c=3

Now

CDN  Node

Originhttp://collector.site.com/beacon.img?a=1&b=2&c=3

Now

CDN  Node

Origin

HTTP/1.1 200 OKExpires: Wed, 19 Apr 2000 11:43:00 GMTCache-Control: no-cache, no-storeLast-Modified: Wed, 21 Jan 2004 19:51:30 GMTContent-Type: image/gifDate: Fri, 20 Jun 2014 12:22:20 GMTServer: ApacheContent-Length: 35

http://collector.site.com/beacon.img?a=1&b=2&c=3

Now  (204)

CDN  Node

Origin

HTTP/1.1 204 No ContentDate: Sat, 21 Jun 2014 23:21:12 GMTServer: Awesome ServerContent-Length: 0

http://collector.site.com/beacon.img?a=1&b=2&c=3

Now

CDN  Node

Origin

Syslog  /  S3  /  FTP/etc

Streamed  W3C  Log

http://collector.site.com/beacon.img?a=1&b=2&c=3

Summary:  CDN  is  an  extension  of  the  app

• Flexible  caching…• …and  uncaching• All  tail  sizes  should  perform  comparably!• Control  over  functionality  at  the  edge• Real  time  interfaces  for  programmability• Logic  at  the  edge• Real-­‐time  visibility• Real-­‐time  analytics

The  future

• Security  (more  now  than  future!)

• Even  more  at  the  edge• More  logic• Other  parts  of  applications• More  delivery  features

Thank  you!hooman@fastly.com