a modern approach to performance monitoring
DESCRIPTION
Akamai Edge 14 - Discussion on RUM, Synthetic and setting realistic and meaningful performance goals.TRANSCRIPT
![Page 1: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/1.jpg)
A Modern Approach to Performance Monitoring
![Page 2: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/2.jpg)
Cliff Crocker@cliffcrockerSOASTAVP Product, mPulse
![Page 3: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/3.jpg)
1. How fast am I?
2. How fast should I be?
3. How do I get there?
![Page 4: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/4.jpg)
Synthetic vs. RUM
![Page 5: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/5.jpg)
What is RUM?Real User Measurement (traditional definition)Synthetic 101Synthetic monitoring (for purposes of this discussion) refers to the use of automated agents (bots) to measure your website from different physical locations.
• A set ‘path’ or URL is defined• Tests are run either adhoc or scheduled and data is collected
![Page 6: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/6.jpg)
What is RUM?Real User Measurement (traditional definition)RUM 101Real User Measurement (RUM) is a technology for collecting performance metrics directly from the browser of an end user.
• Involves instrumentation of your website via JavaScript
• Measurements are fired across the network to a collection point through a small request object (beacon)
<JS>
<beacon>
![Page 7: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/7.jpg)
RUM
Cast a wide net• Identify key areas of concern• Understand real user impact• Map performance to human behavior
& $$
Synthetic
Diagnostic tool• Identify issues in a ‘lab’/remove
variables• Reproduce a problem found with RUM http://www.flickr.com/photos/84338444@N00/with/3780079044/
http://www.flickr.com/photos/ezioman/
![Page 8: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/8.jpg)
The Early Days of RUM
• Round-trip time• Start/stop timers via JavaScript• Early contributors:• Steve Souders/Episodes• Philip Tellis/Boomerang.js• Both widely in use today
![Page 9: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/9.jpg)
Navigation TimingWhat gives RUM a kick
![Page 10: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/10.jpg)
Browser Support for Navigation Timing
!
![Page 11: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/11.jpg)
11© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Navigation TimingExamples
DNS: Domain Lookup Time
function getPerfStats() { var timing = window.performance.timing; return { dns: timing.domainLookupEnd - timing.domainLookupStart};}
![Page 12: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/12.jpg)
12© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Navigation TimingExamples
TCP: Connection Time to Server
function getPerfStats() { var timing = window.performance.timing; return { connect: timing.connectEnd - timing.connectStart};}
![Page 13: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/13.jpg)
13© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Navigation TimingExamples
TTFB: Time to First Byte
function getPerfStats() { var timing = window.performance.timing; return { ttfb: timing.responseStart - timing.connectEnd};}
![Page 14: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/14.jpg)
14© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Navigation TimingExamples
Base Page
function getPerfStats() { var timing = window.performance.timing; return { basePage: timing.responseEnd - timing.responseStart};}
![Page 15: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/15.jpg)
15© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Navigation TimingExamples
Front-end Time
function getPerfStats() { var timing = window.performance.timing; return { frontEnd: timing.loadEventStart - timing.responseEnd};}
![Page 16: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/16.jpg)
16© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Navigation TimingExamples
Page Load Time
function getPerfStats() { var timing = window.performance.timing; return { load: timing.loadEventStart - timing.navigationStart};}
![Page 17: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/17.jpg)
Measuring AssetsYour page is composed of many different parts
• Strength of synthetic• Full visibility into asset performance• Images• JavaScript• CSS• HTML
• A lot of which is served by third-parties• CDN!
![Page 18: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/18.jpg)
Object Level RUMGoing beyond page level metrics
![Page 19: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/19.jpg)
Browser Support for Resource Timing
![Page 20: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/20.jpg)
CORS: Cross-Origin Resource Sharing
Timing-Allow-Origin = "Timing-Allow-Origin" ":" origin-list-or-null | "*"
Start/End time only unless Timing-Allow-Origin HTTP Response Header defined
![Page 21: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/21.jpg)
Resource Timing
var rUrl = ‘http://www.akamai.com/images/img/cliff-crocker-dualtone-150x150.png’;var me = performance.getEntriesByName(url)[0];var timings = { loadTime: me.duration, dns: me.domainLookupEnd - me.domainLookupStart, tcp: me.connectEnd - me.connectStart, waiting: me.responseStart - me.requestStart, fetch: me.responseEnd - me.responseStart}
Measuring a single resource
![Page 22: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/22.jpg)
Resource TimingOther uses
• Slowest resources• Time to first image• Response time by domain• Time a group of assets• Response time by initiator type (element type)• Cache-hit ratio for resources
For examples see: http://www.slideshare.net/bluesmoon/beyond-page-level-metrics
![Page 23: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/23.jpg)
Resource TimingCool projects
• PerfMap - https://github.com/zeman/perfmap• Mark Zeman
• Waterfall.js - https://github.com/andydavies/waterfall• Andy Davies
![Page 24: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/24.jpg)
1. How fast am I?
2. How fast should I be?
3. How do I get there?
![Page 25: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/25.jpg)
Picking a NumberHow do you set front-end performance goals?
• Industry benchmarks?
• Apdex?• Analyst report?• Case studies?
![Page 26: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/26.jpg)
“Synthetic monitoring shows you how you relate to your competitors, RUM shows you how you relate to your customers.”
– Buddy Brewer
![Page 27: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/27.jpg)
BenchmarkingHow you relate to the competition
![Page 28: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/28.jpg)
BenchmarkingSOASTA mPulseUX
• http://soasta.com/mpulseUX
![Page 29: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/29.jpg)
BenchmarkingImportant to remember that it’s not always about Page Load
• Page construction• Requests• Images• Size
• Other important metrics• Speedindex• Start Render• PageSpeed/Yslow scoring
![Page 30: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/30.jpg)
Performance is a Business Problem
![Page 31: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/31.jpg)
Yahoo! - 2008Slower pages cause increase in abandonment
Increase of 400ms causes 5-9% increase in user abandonment
http://www.slideshare.net/stubbornella/designing-fast-websites-presentation
![Page 32: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/32.jpg)
Shopzilla - 2009Faster pages increase site conversion rate
A reduction in Page Load time of 5s increased site conversion 7-12%!
http://assets.en.oreilly.com/1/event/29/Shopzilla%27s%20Site%20Redo%20-%20You%20Get%20What%20You%20Measure%20Presentation.ppt
![Page 33: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/33.jpg)
Walmart - 2012Impact of slow pages on conversion rate
http://minus.com/msM8y8nyh#1e
SF WebPerf – 2012Up to 2% conversion drop for every additional second
![Page 34: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/34.jpg)
So What?
![Page 35: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/35.jpg)
SIMULATION
![Page 36: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/36.jpg)
![Page 37: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/37.jpg)
![Page 38: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/38.jpg)
How Fast Should You Be?Key takeaways
• Use synthetic measurement for benchmarking your competitors
• Understand how fast your site needs to be to reach business goals/objectives with RUM
• You must look at your own data
![Page 39: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/39.jpg)
1. How fast am I?
2. How fast should I be?
3. How do I get there?
![Page 40: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/40.jpg)
Real users are not normal
![Page 41: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/41.jpg)
![Page 42: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/42.jpg)
Page Load TimesAll Countries, User Agents, Carriers, Operating Systems
2.76s – Median
10.45s – p95
17.26s – p98
![Page 43: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/43.jpg)
Platform Median 95th Percentile
98th Percentile
Mobile 3.62s 12.53s 20.04s
Desktop 2.44s 9.31s 15.86s
Page Load TimesBy Platform
![Page 44: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/44.jpg)
Page Load TimesBy Operating System
OS Median 95th Percentile
98th Percentile
Windows 7 2.41s 9.29s 15.89s
Mac OS X/10 2.30s 8.11s 13.45s
iOS7 3.27s 10.64s 15.79s
Android 4 4.06s 14.30s 27.93s
iOS8 3.53s 11.54s 19.72s
Windows 8 2.67s 10.75s 18,74s
![Page 45: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/45.jpg)
Other FactorsAll Countries, User Agents, Carriers, Operating Systems
• Geography• User Agent• Connection Type• Carrier/ISP• Device Type
![Page 46: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/46.jpg)
Not All Pages are Created EqualTop of funnel impact (Browse Pages)
For a typical eCommerce site, conversion rate drops by up to
50% when “browse” pages increase from 1
to 6 seconds
![Page 47: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/47.jpg)
Not All Pages are Created EqualBottom of funnel impact (Checkout pages)
However, there is much less
impact to conversion
when “checkout”
pages degrade
![Page 48: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/48.jpg)
How Do I Get There?Key takeaways
• Focus on the highest value opportunities/demographics
• Identify key pages that have the most impact on your KPIs
• Prioritize based on reducing friction within the funnel or critical path
![Page 49: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/49.jpg)
Thank You!
![Page 50: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.us/reader033/viewer/2022042713/547e61cfb47959b1508b4b33/html5/thumbnails/50.jpg)
AttributionsImages
1. Modern (Title Slide): https://www.flickr.com/photos/looking4poetry/3477854720/in/photolist-6ijV3q-9t3sjb-cj8VWE-MNjPA-4yBqug-2rk5he-4nZVzJ-4yE45J-9t3Fvj-2rptr5-69ymdr-6XVTH-6QZ4dg-9t3HN5-8geUkB-6oEXs5-eH9zS-H2XTt-cKdJvb-7GNKWx-cKdKZd-5Rw3qt-jfT5Dx-Jfiuy-bTop6R-2m8kAB-Jfiuu-7E8eMf-9jqD6-9hYAvd-Jfius-gsqr7U-7KiNAZ-8kzG9V-euXdb-44qJNN-47jBuY-nsxZrZ-7cGjQL-4cfHKq-cGzbSC-aQtWPR-8y3sR3-6okjAW-5A7BSC-6aRsvh-eSMHkX-kxvdyK-9t3udu-c8EkYu
2. Man vs. Machine: https://www.flickr.com/photos/eogez/3289851965/in/photolist-61HmnB-9nkQu3-dfinF1-4GjK24-4GjL6Z-4GoV6C-dibVCU-4GoTXo-4GjF3k-4GjH7Z-4GoR6h-ek6eT9-ek6eUU-4fADUx-4fEDV9-65JA6W-65Jsxu-65JDnj-4fAE12-65JDTh-65JBZu-65Ek9M-65JtDu-65JuiN-65JzFS-65EfiH-65Jv63-65JxX7-65Enyc-65Eh4c-65Jxjm-65JwmU-65EhUP-65JCML-65EjxP-65JBQf-65Ee4Z-65JD4b-65JBF1-65Ec1r-65EkPB-65Jy4o-65Eg96-65JAjb-65Emfe-65Efoz-65JBam-65Eniv-65JCrU-65Eed2
3. https://www.flickr.com/photos/bradybd/2818154005/
4. https://www.flickr.com/photos/perspective/149321089/
5. Dartboard: http://en.wikipedia.org/wiki/Darts#mediaviewer/File:Darts_in_a_dartboard.jpg
References:
6. http://calendar.perfplanet.com/2011/a-practical-guide-to-the-navigation-timing-api/
7. http://www.slideshare.net/bluesmoon/beyond-page-level-metrics