altitude sf 2017: optimizing your hit rate
Post on 22-Jan-2018
276 Views
Preview:
TRANSCRIPT
Optimizing Your Cache Hit Ratio
Léon BrocardSales Engineer | Fastly
Paolo AlvaradoCustomer Support Manager | Fastly
Léon Brocard
Systems Engineer
Fastly London
Paolo Alvarado
Customer Support Manager
Fastly Tokyo
Cache hit ratioWhy do we cache?Caching on FastlyHTTP cachingOverriding cachingCaching patternsFuture
This is a talk workshop
Blue slidemeans
interactive
Cachehit
ratio
Where is“Cache hit ratio” on
https://www.fastly.com/
https://manage.fastly.com/
altitude2017chr@example.comchr4l1fe
Name & Domainhitrate2017<initials>.global.ssl.fastly.net
Address35.197.9.99
No TLS, Port3000
Name & Domainhitrate2017lb.global.ssl.fastly.net
Address35.197.9.99
No TLS, Port3000
http://hitrate2017lb.global.ssl.fastly.net/
Why do we cache?
Decrease pageload time
Level Capacity Latency
Can on my desk
1 can ~1s
My fridge 10 cans ~30s
Local shop 500 cans ~5m
Supermarket 10k cans ~90m
Level Capacity Latency
Can on my desk
1 can ~1s
My fridge 10 cans ~30s
Local shop 500 cans ~5m
Supermarket 10k cans ~90m
Level Capacity Latency
Cash in my wallet
$50 ~1s
Local ATM $200 ~5m
Bank $1,000 ~20m
Level Capacity Latency
Plate 1 serving ~1s
Serving dish 10 servings ~1m
Kitchen 100 servings ~5m
Level Capacity Latency
L1 cache 64 KiB per core ~1ns
L2 cache 256 KiB per core ~4ns
L3 cache Up to 2 MiB per core, shared
~20ns
Main memory Up to 64 GB ~100ns
Level Capacity Latency
Browser ~Megabytes ~0.1ms
CDN ~Gigabytes ~5ms
Origin ~Terabytes ~500ms
Cache?
L'écureuil roux cache ses provisions pour l'hiver
Hit Miss
Scenario Origin requests / sec
EC2 instances
No CDN 2,000 40
50% CHR 1,000 20
80% CHR 400 8
90% CHR 200 4
95% CHR 100 2
Representativetime periodtraffic
Why is latency so important?
Wi-Fi Mobile
HTTPTCP/IP
GET /technology/2016/jul/15/how-the-internet-was-invented-1976-arpa-kahn-cerf HTTP/1.1Host: www.theguardian.com
HTTP/1.1 200 OKContent-Type: text/html; charset=utf-8Content-Length: 110502
<!DOCTYPE html>...
Visithitrate2017lb.global.ssl.fastly.net
View ▶ Developer ▶ Developer Tools
Network tab
Reload, ⌘R, shift-⌘R
c
1.5s 2.0s 3.5s 21.0s
www.webpagetest.orgURL: docs.fastly.comEC2 location with low pending testsConnection DSL, capture video, keep test private
Here’s one I prepared earlier:
http://fastly.us/2sybe4q
Back to developer toolsRight click on table headingResponse headers ▶ Add “Cache-Control”Response headers ▶ Manage header columns ▶ Add custom header → “X-Cache”
if users double their bandwidth without reducing their RTT significantly, the effect on Web Browsing will be a minimal improvement. However, decreasing RTT, regardless of current bandwidth always helps make web browsing faster
How can we decrease RTT?
How can we decrease RTT?
latency?
Caching
Visithttps://www.fastly-debug.com
/
HTTPcaching
1996 RFC 1945 HTTP/1.0
1997 RFC 2068 HTTP/1.1
1999 RFC 2616 HTTP/1.1
2014 RFC 7230 HTTP/1.1: Message Syntax and Routing
2014 RFC 7231 HTTP/1.1: Semantics and Content
2014 RFC 7232 HTTP/1.1: Conditional Requests
2014 RFC 7233 HTTP/1.1: Range Requests
2014 RFC 7234 HTTP/1.1: Caching
2014 RFC 7235 HTTP/1.1: Authentication
2015 RFC 7540 HTTP/2
Fresh Expired
Last-Modified: Fri, 13 Nov 2015 14:11:55 GMT
If-Modified-Since: Fri, 13 Nov 2015 13:58:51 GMT
HTTP/1.1 304 Not Modified
ETag: “3e8d1bf998b31cb811264f4789eac005”
If-None-Match: “3e8d1bf998b31cb811264f4789eac005”
HTTP/1.1 304 Not Modified
Visithitrate2017lb.global.ssl.fastly.net
Network tabBrowser cachemax-age=60 tabFirst time. Second time.
Cache-Control: max-age=60Date: Thu, 25 May 2017 10:16:09 GMTExpires: Thu, 25 May 2017 10:16:58 GMT
Cache-Control: max-age=1800
Cache-Control: s-max-age=1800, max-age=60Surrogate-Control: max-age=1800
Visithitrate2017lb.global.ssl.fastly.net
Network tabBrowser cache⌘RCache-Control:(none) / private / max-age=60
Tool time
Install Chrome Extension “Browser Cache Hit Ratio”...
https://fastly.us/browser-cachehitratio
Install Chrome Extension “Fastly Cache Hit Ratio”...
https://fastly.us/cachehitratio
Install Chrome Extension “Modheader”...
https://fastly.us/modheader
Visithitrate2017lb.global.ssl.fastly.net
Break
What’s myTTL again?
Changingcaching
On the origin with
HTTP headers
Override in Fastly
No Cache-Control header
Fastly Service ▶ Configuration ▶ Clone active ▶ Settings ▶ Fallback TTL ▶ 10 ▶ Save ▶ Activate ▶ Purge ▶ Purge All ▶ Purge
Cache-Control: private
Cache-Control: max-age=60
Varnish Configuration Language
defaultTTL?
if ( beresp.http.Expires || beresp.http.Surrogate-Control ~ "max-age" || beresp.http.Cache-Control ~"(s-maxage|max-age)" ) { # keep the ttl here } else { # apply the default ttl set beresp.ttl = 10s;}
if (req.url == "/dogs/max-age=60") { set beresp.http.Cache-Control = "max-age=10";}
if (req.url == "/dogs/max-age=60") { set beresp.ttl = 10s; set beresp.http.Cache-Control = "max-age=10";}
sub vcl_fetch { if (beresp.http.Cache-Control) { if (req.url.path ~ "^/users/flair/") { set beresp.http.Cache-Control = "public, max-age=180"; } else { set beresp.http.Cache-Control = "private"; }}
Here’s an example of where VCL comes in very handy. Recently we deployed .NET 4.6.2 which had a very nasty bug that set max-age on cache responses to over 2000 years. The quickest way to mitigate this for all of our services affected was to override that cache header as-needed at the edge. As I write this, the following VCL is active:
What’s in a cache key?
req.http.hostThe incoming request’s host header
req.urlThe full URL including query string
#####GENERATION#####A generation number used for purging all
What do these URLs have in common?
● /search?query=dogs&page=1● /search?page=1&query=dogs
/search?query=dogs&page=1Fastly-Debug-Digest: 9ff1c58f06cac2fd3a48f7c105da6e1d035c770c635371f2ddabdb987818e36c
/search?page=1&query=dogsFastly-Debug-Digest:9ff1c58f06cac2fd3a48f7c105da6e1d035c770c635371f2ddabdb987818e36c
The uncacheable
“Dynamic content is really interesting”—Hooman 2014
Classically, dynamic content was uncacheable because it required interaction with a backend.
How can we cache dynamic
content?
Set-Cookie:Cache-Control: private
How do we force an update for cached
content?
Purging
Single URL PurgeInstant Purge an individual URL.
Surrogate-Key PurgeInstant Purge a particular service of items tagged with a Surrogate-Key.
Purge AllInstant Purge everything from a service.
Offloading from origins
HTTPS everywhere
http://hitrate2017lb.global.ssl.fastly.net/301 Moved Permanently redirect to:https://hitrate2017lb.global.ssl.fastly.net/
HIT-SYNTH
Strict-Transport-Security: max-age=31536000
HSTS Preload List
Serving Stale
What does a negative number here mean?
“Worst-case scenario, we can always ghost-serve. Which actually happened: our Rolling Stone legacy origin was down for three days; the only reason readers could still visit our site without noticing the problem was because we used Fastly’s stale-if-error.”
— Chris Boylan, Director of Engineering,Wenner Media
How can we purge while still
offloading?
Soft PurgeSingle URL PurgeSurrogate-Key Purge
Soft Purging sets an object’s TTL to 0s, forcing revalidation. For best results, use in combination with stale-while-revalidate and/or stale-if-error
Levels of caches
× =
× =
50%CHR
50%CHR
× = 75%CHR
50 50
25 25
90%CHR
90%CHR
× = 99%CHR
90 10
9 1
× =
When the royal baby George Alexander Louis was born in 2013, the Guardian's traffic spiked from an average of 400 requests per second to over 1,000 requests per second. Fastly shielded the Guardian's origin servers from the hundreds of thousands of new requests pouring in, maintaining site uptime and delivering consistent performance for readers around the world. The company’s software architect, Matthew O’Brien, said that it’s difficult to predict news events, but Fastly helps them prepare accordingly.
Cache in browser and Fastly for different
amount of time?
Cache-Control: privateSurrogate-Control: max-age=60, stale-while-revalidate=5, stale-if-error=86400
Cache-Control: max-age=5Surrogate-Control: max-age=3600
“Fastly effectively handled the higher levels of traffic we experienced during the Apple Watch announcement.” Using Fastly’s stale-while-revalidate feature to serve readers slightly outdated content while the cached content was refreshed, WIRED knew readers would never see errors. Zack described stale-while-revalidate as one of Fastly’s “most exciting features.” He said, “It allowed us to successfully serve the live blog while constantly updating its content, ensuring our readers had access to the freshest content without seeing errors."
— Zack Tollman, Lead Engineer at WIRED
Observing cache hit ratio
X-Cache: HITX-Cache-Hits: 649406
X-Cache: HIT, HITX-Cache-Hits: 649406, 922
fastly_info.state
Jun 08 14:27:24 cache-ord1722 papertrail: MISS-CLUSTER "-" [08/Jun/2017:14:27:23 +0000] GET /recipes/?show=Beef%20brochettes HTTP/1.1 200 26
Logging atthe edge
Chris JackelSystems Engineer | Fastly
vcl_hashvs
Vary
Hash Vary
Accept-Encoding: gzip, deflate, br
Content-Encoding: gzipVary: Accept-Encoding
Future
Cache-Control: max-age=31536000, immutable
HTTP/2 enables a more efficient use of network resources and a reduced perception of latency by introducing header field compression and allowing multiple concurrent exchanges on the same connection
Service WorkersThis specification describes a method that enables applications to take advantage of persistent background processing, including hooks to enable bootstrapping of web applications while offline
The power of the network
Andrew BettsPrincipal Developer Advocate | Fastly
Emerging and future web trends
Alex RussellSoftware Engineer | Google
Summary
Cache hit ratioWhy do we cache?Caching on FastlyHTTP cachingOverriding cachingCaching patternsFuture
Run live experiments
Thanks for participatingLéon & Paolo
top related