tips for going fast in a slow world: michael may at oscon 2015

CACHE MONEY: Tips for going FAST in a slow world

Michael May API Engineer @ohaimmay 2015-07-22

Real-time Content Delivery Network Leverage fork of Varnish Cache sub-millisecond TTFB Enable, not constrain

Physical Realities

ALL communication is bounded by the speed of light

C(vac) = 186,000 miles/s C(fiber) = 124,000 miles/s

c(air) ~ c(vac)

We’re really impatient (when it comes to cat gifs, at least)

The NASA Problem

Get pics from Deep Space to Earth 1 kb/sec bandwidth

Single (compressed) 2.5Mb image takes ~1 hour Complete sensor dataset won’t arrive until 2016!

*source: http://www.planetary.org/blogs/emily-‐lakdawalla/2015/01300800-‐talking-‐to-‐pluto-‐is-‐hard.html

New Horizons

http://www.planetary.org/blogs/emily-lakdawalla/2015/01300800-talking-to-pluto-is-hard.html

Credit: NASA, Expedition 24 Crew

We have the same problem on Earth!

Credit: NASA, Expedition 24 Crew

mmay:~ $ ping fast.mmay.net

PING fast.mmay.net: 56 data bytes64 bytes received: icmp_seq=0 ttl=44 time=273.722 ms64 bytes received: icmp_seq=1 ttl=43 time=347.763 ms64 bytes received: icmp_seq=2 ttl=44 time=276.129 ms64 bytes received: icmp_seq=3 ttl=43 time=369.563 ms64 bytes received: icmp_seq=4 ttl=44 time=269.116 ms

--- fast.mmay.net ping statistics —6 packets transmitted, 6 packets received, 0.0% packet loss

round-trip min/avg/max/stddev = 269.116/293.231/503.991/77.754ms

ATL to SYD RTT (9k miles)

Moving bits around the world takes measurable time


Moving bits around the ^world takes measurable time


universe

ATL to SYD RTT (what we want)mmay:~ $ ping fast.mmay.net

PING fallback.global-ssl.fastly.net (23.235.39.249): 56 data bytes64 bytes from 23.235.39.249: icmp_seq=0 ttl=57 time=22.411 ms64 bytes from 23.235.39.249: icmp_seq=2 ttl=57 time=30.053 ms64 bytes from 23.235.39.249: icmp_seq=3 ttl=57 time=23.592 ms64 bytes from 23.235.39.249: icmp_seq=4 ttl=57 time=23.180 ms64 bytes from 23.235.39.249: icmp_seq=5 ttl=57 time=24.993 ms

--- fallback.global-ssl.fastly.net ping statistics ---6 packets transmitted, 6 packets received, 0.0% packet lossround-trip min/avg/max/stddev = 22.411/27.042/38.025/5.510 ms

We’re not going to beat physics

Credit: US Navy / CC-‐BY-‐2.0

We’re not going to beat physics

Credit: US Navy / CC-‐BY-‐2.0

But we will be clever engineers

> “Middle Mile” Optimizations

> “Middle Mile” Optimizations

> Global Cache

Will it cache?

Image Credit: © Ryan Notch / www.areographers.com / CC-‐SA-‐3.0

STATIC

ETERNALLY

STATIC

TRULY

DYNAMIC

EVENT

DRIVEN

“Event Driven” Content

> Valid for > 0 seconds (pseudo-static?)

> Changes unpredictably

> That change requires immediate update

mmay:~ $ ping new-horizons.nasa.space

PING new-horizons.nasa.space: 56 data bytes1 byte received: time=16200s (4.5 hours)

--- ping new-horizons.nasa.space ping statistics —1 packets transmitted, 1 packets received

SFO to PLUTO RTT (3 billion mi)

Web Perf 101

Critical path optimization / use Google PageSpeed

Rendering optimizations

Compress/Minify text (html, js, css), images, fonts

Optimize HTTP Caching Strategies!!!!

*Optimize HTTP Caching Strategies

Varnish

Varnish

HTTP Reverse Proxy Cache

Is my Varnish Working?

Is my Varnish Working?mmay:~ $ curl -svo /dev/null https://app.fastly.com* Connected to app.fastly.com > GET / HTTP/1.1> User-Agent: curl> Host: app.fastly.com>< HTTP/1.1 200 OK< Connection: keep-alive< Via: 1.1 varnish< X-Served-By: cache-atl6230-ATL< X-Cache: MISS< X-Cache-Hits: 0< Vary: Accept-Encoding[data not shown]* Connection #0 to host app.fastly.com left intact


I’ll be talking to ya


Hey, I was proxied

Is my Varnish Working?mmay:~ $ curl -svo /dev/null https://app.fastly.com* Connected to app.fastly.com > GET / HTTP/1.1> User-Agent: curl> Host: app.fastly.com>< HTTP/1.1 200 OK< Connection: keep-alive< Via: 1.1 varnish< X-Served-By: cache-pdx999-PDX< X-Cache: HIT< X-Cache-Hits: 10< Vary: Accept-Encoding[data not shown]* Connection #0 to host app.fastly.com left intact

Is my Varnish Working?mmay:~ $ curl -svo /dev/null https://app.fastly.com* Connected to app.fastly.com > GET / HTTP/1.1> User-Agent: curl> Host: app.fastly.com>< HTTP/1.1 200 OK< Connection: keep-alive< Via: 1.1 varnish< X-Served-By: cache-pdx999-PDX< X-Cache: HIT< X-Cache-Hits: 10< Vary: Accept-Encoding[data not shown]* Connection #0 to host app.fastly.com left intact

Cool cache stuff


Cache-Control: public, max-age=10

Is the object in the cache valid?

Is the object in the cache valid?aka Revalidation

HTTP Response with ETag

HTTP Request with If-None-Match

Cache-Control: First Request

Cache-Control: Grace Mode

stale-if-error / stale-while-revalidate

Cache-Control: max-age=3600, s-maxage=864000, stale-while-revalidate=3600, stale-if-error=3600

Back in the day, the server was responsible for rendering pages

AJAX

Extract server-side rendering of private content into APIs

Fetch & render private data client-side

Cache API for increased performance

API Caching with event-driven purge

Set Cache-Control with a reasonably long TTL



Set Surrogate-Key(s) with unique cache key(s)



Set Surrogate-Key(s) with unique cache key(s)

Invalidate cached responses by Surrogate-Key purge

Example response handlers with API CachingGET /content.json { - Validation - DB Lookup - Set Cache-Control Header - Set ‘Surrogate-Key: my_cache_key’ Header - Render response}

PUT /content.json { - Validation - Update DB - Purge ‘my_cache_key’ (e.g. curl -XPOST https://api.fastly.com/purge/my_cache_key) - Render response

AJAX “like” request flow w/ Varnish

Varnish

Request flow with Varnish Synthetic

Varnish

Synthetic Response VCL

synthetic {"OK"};

Synthetic Response VCL for “like”

hello world

sub vcl_recv {

if (req.url ~ "^/like" && req.request == "POST") { error 702 "OK"; }}


hello worldsub vcl_error {

if (obj.status == 702) { set obj.status = 200; set obj.http.Content-Type = “text/plain;” synthetic {"OK"}; }

return (deliver);}


hello world

sub vcl_log {

if (req.url ~ "^/like" && req.method == "POST") { log {"syslog “like :: “} req.url {" "} resp.status; }}

Caching and Cookies

Set-Cookie

Sent by server “Cookie” sent by client

If cookies in response, not cached by default

Vary’ing on Cookie is unwise

Vary

Specifies which headers the cache will take into account on object lookup. (outside of Host and Path)

Out of a sample of 100,000 requests, there were 8,000 different User-Agents!

Caching when cookies present

1. Remove cookie and save in temp variable

2. Do normal cache lookup

3. Set cookie from temp var right before deliveryhello world

hello world

Caching when cookies present (VCL)

// Request receivedsub vcl_recv {

// If request includes Cookie, save temporarily// & remove for cache lookup

if (req.http.Cookie ~ "mycookie=") { set req.http.Tmp-Set-Cookie = req.http.Cookie; unset req.http.Cookie; } else { set req.hash_always_miss = true; }}

Caching when cookies present (VCL)// Cache miss, fetch from originsub vcl_fetch {

// if origin returns Set-Cookie, save temporarily // & remove so response can be cached if (beresp.http.Set-Cookie) { set req.http.Tmp-Set-Cookie = beresp.http.Set-Cookie; unset beresp.http.Set-Cookie; }}

// One last thingsub vcl_deliver {

// Include Set-Cookie in response to client if (req.http.Tmp-Set-Cookie) { set resp.http.Set-Cookie = req.http.Tmp-Set-Cookie; }}

Wrapping Up

Wrapping Up

c is our speed limit, moving data around takes time

Wrapping Up

c is our speed limit, moving data around takes time

CDNs are more than a dumb static content cache

Wrapping Upc is our speed limit, moving data around takes time

CDNs can be more than a dumb static content cache

Cache “event driven” content and script logic at the edge using VCL

Wrapping Upc is our speed limit, moving data around takes time

CDNs can do more than be a static content cache

Cache “event driven” content and script logic at the edge using VCL

Increase that CACHE HIT RATIO!

tips for going fast in a slow world: michael may at oscon 2015

Technology

data bytes

ms atl

measurable time user

space ping statistics

space ping newhorizons

new horizons

nasa problem

cache money