table of contents · inbound marketing leads will be lost when your site underperforms and...
Post on 18-Jul-2020
0 Views
Preview:
TRANSCRIPT
TABLE OF CONTENTS
Defining Digital Experience Monitoring .................................................................................... 2
Understanding the Impact of Digital Experiences .................................................................... 4
50 Latest Mobile Performance Stats .................................................................................... 5
Not Meeting Online User Expectations Is Costly .................................................................. 8
Meeting And Exceeding User Expectations Drives More Business! ..................................... 9
Identifying the Causes of Poor Digital Experiences ............................................................... 10
Measures To Reduce Your Risk ......................................................................................... 14
Discovering Digital Experience Issues ................................................................................... 15
5 Dem Strategies ................................................................................................................ 15
Help Desk, Tech Support, and Social Media ....................................................................... 16
Real User Measurement (RUM) .......................................................................................... 17
Application Monitoring ....................................................................................................... 18
Cloud-based Synthetic Monitoring ..................................................................................... 19
On-premise Synthetic Monitoring ....................................................................................... 20
Choosing The Right Monitoring Strategy ............................................................................... 21
Which Tool is Right For Me? ............................................................................................... 21
Why You Must Have Synthetic Monitoring In Your Mix ...................................................... 23
Monitoring Checklist To Get You Going ................................................................................. 24
Digital Experience Monitoring Selection Guide ...................................................................... 25
References ............................................................................................................................. 26
The Ultimate Guide to DEM
Digital Experience Monitoring 2
DEFINING DIGITAL EXPERIENCE MONITORING
End-User Experience
End-user experience describes the quality of all the interactions and engagements
that the users of an organization’s application or service have with that application
or service. These users can be external such as customers, partners, suppliers or
site visitors, or internal, such as employees.
End-User Experience Monitoring (EUM)
End-user experience monitoring describes various technologies used to monitor
application and service performance levels as the end user experiences them. This
can include web page or application wait times, load times, transaction response
times, and availability, all from the end users’ perspective. The primary
technologies for end-user experience monitoring are synthetic monitoring, which
automates interactions, including multi-step transactions, with a website or
application to constantly test how that application performs, and real user
measurement, which shows how actual users navigate your site or applications
and how their behavior is affected by your site or applications’ performance.
EUM is a critical part of any application performance management strategy. But
while traditional APM tools focus on back-end metrics, like server processing time,
database operations, CPU utilization, memory consumption, IOPS, or code
The Ultimate Guide to DEM
Digital Experience Monitoring 3
execution time, EUM allows you to connect the dots between those metrics and
what your end users are actually experiencing while using your site or application.
Digital Experience Monitoring
EUM is a subset of an emerging technology known as Digital Experience
Monitoring (DEM). While EUM looks specifically at the human end-user or
customer, DEM looks at the experience of all digital agents—human and
machine—as they interact with enterprises’ application and service portfolios.
As Internet of Things technologies continue to mature, these machine digital
agents will become increasingly common, will interact with applications
continuously, and will not be limited to internal users or customers. Indeed, they
may not even transact with the organization’s systems as human users do, but will
still generate valuable data that’s useful to suppliers, partners, and observers in
addition to the enterprise itself. These machine digital agent interactions must be
monitored to provide the most complete view of application and service level
quality.
The Ultimate Guide to DEM
Digital Experience Monitoring 4
UNDERSTANDING THE IMPACT OF DIGITAL EXPERIENCES
A one-second delay in web page load time could cost Amazon up to $1.6 billion a year.
Just 4/10th of a second delay decreased Google searches by 8 million a day.1
Fast and reliable digital experiences make for happy users (and employees)
and repeat business, but they’re not built overnight, nor is the work ever
complete. Performance is a journey (not a destination), so creating a
company “culture of quality performance” is key to building and maintaining
top performing sites.
Building such a culture isn’t easy, but it starts with understanding and
communicating the importance of reducing latency, maintaining uptime, and
improving speed on every release.
The Ultimate Guide to DEM
Digital Experience Monitoring 5
50 MOBILE PERFORMANCE STATS YOU NEED TO KNOW
Your customers’ experience on mobile is more important than ever.
These 50 mobile performance stats say it all.
MOBILE PERFORMANCE
1. Mobile pages make an average of 214 server requests, and nearly half of all server requests are
ad-related. 2
2. The average weight of the content on mobile sites is 1.49 MB, which takes 7 seconds to load over
3G connections. 3
3. The average load time for mobile sites is 19 seconds over 3G connections. 4
4. One out of two people expect a page to load in less than 2 seconds. 5
5. 85% of mobile users expect pages to load as fast as or faster than they load on the desktop. 6
6. Mobile sites that load in 5 seconds earn up to 2x more mobile ad revenue than those whose sites
load in 19 seconds. 7
7. Sites that load in 5 seconds vs 19 seconds observed 25% higher ad viewability. 8
8. Sites that load in 5 seconds vs 19 seconds observed 70% longer average sessions. 9
9. Sites that load in 5 seconds vs 19 seconds observed 35% lower bounce rates. 10
10. Latency for major mobile carriers in the US ranges from 340 to 362 milliseconds per request. 11
11. In 2011, almost half of all pages served to mobile contained fewer than 25 total requests. 12
12. Today, 1 out of 5 pages contains 100 or more resource requests and more than half of all pages
contain 50 or more requests. 13
13. The average page served to mobile has grown by 203% since 2011. 14
14. In 2011, only 2% of all pages served to mobile devices used custom fonts. In 2015, that number
has increased 48%.15
15. The average page served to mobile is 3X bigger than it was four years ago. 16
The Ultimate Guide to DEM
Digital Experience Monitoring 6
MOBILE USAGE
16. 75% of Internet users went online via a mobile device at the end of 2015. 17
17. In many countries, including the US, more searches take place on mobile devices than on
computers. 18
18. The percentage of people making mobile purchases steadily increased to 30% in Q4-2015 (24%
on a phone and 6% on a tablet) and the frequency of mobile purchases increased 35%.19
19. 60% of customers seek discounts and sales, 36% seek product reviews, and 35% seek product
information, via mobile ads. 20
20. 57% of tablet users conduct product searches at least once a week, compared to 37% of
desktop users. 21
21. By 2020, there will be roughly 6.1 billion mobile users. 22
22. Wi-Fi and mobile-connected devices will generate 68% of all Internet traffic by 2017. 23
23. There are more mobile Internet users than desktop Internet users; 52.7% of global Internet users
access the Internet via mobile, and 75.1% of US Internet users access the Internet via mobile. 24
COST OF POOR MOBILE PERFORMANCE
24. 46% of people say that waiting for pages to load is what they dislike the most when browsing
the web on mobile devices. 25
25. 53% of mobile site visits are abandoned if pages take longer than 3 seconds to load. 26
26. 65% of consumers say that a poor online experience has a direct impact on their opinion of a
brand. 27
27. 39% of mobile users are not happy with their online experience due to slow pages and mobile
site freezes/crashes. 28
28. 46% of mobile users would abandon a page if it did not load within 3 seconds. 29
29. Tablet users are 33% less likely to purchase from a company online if they experience poor site
performance, 46% will go to competitor websites, and 35% are less likely to visit the problematic
website on any platform. 30
30. Every one-second delay in web page load time could lead to $1.6B in annual losses for major
online merchants. 31
31. 52% of customers are less likely to engage with a company due to a bad mobile experience. 32
32. 30% of dissatisfied mobile shoppers say they will never return. 33
The Ultimate Guide to DEM
Digital Experience Monitoring 7
33. 36% cited slow load times as the cause of abandonment on mobile travel sites. 34
M-COMMERCE
34. B2C mobile commerce sales in the US are valued at an estimated $83.93 billion. 35
35. eMarketer estimates US retail m-commerce sales will reach $123.13 billion in 2016. 36
36. In 2015, mobile influenced over $1 trillion in retail sales. 37
37. 27% of US smartphone owners purchased via a retail mobile app. 38
38. Nearly a quarter of respondents said they have between six and ten retail apps on their
smartphones as of April 2016. 39
39. More than half of all time spent on retail sites takes place on a mobile device. 40
40. Mobile shopping cart abandonment rates are higher (at 97%) than desktop shopping cart rates
(at 70 – 75%).41
41. 82% of people consult their phones regarding a purchase they’re about to make in a store. 42
42. 30% of all online purchases took place on mobile (24% on a phone, 6% on a tablet). 43
43. By 2019, 25% of all global online travel bookings will be from mobile devices. 44
44. 50% of all online travel bookings in the US are expected to occur via mobile device in 2016, yet
only 24% of the top 100 mobile travel sites loaded in the ideal time of 4 seconds or less across
devices. 45
INVESTMENT IN MOBILE
45. Only 14% of companies Forrester surveyed use mobile to transform their customer experiences.
46
46. 77% of retailers have sites optimized for smartphones. 47
47. Only 25% of companies will fully integrate mobile into their overall business strategies to
transform their customer experience. 48
48. In 2014, retailers spent an average of $1.2 million on smartphone investments and $550,000 on
tablet investments. 49
49. Mobile search ads were estimated to be $12.85 billion in 2015, over 50% of the search market. 50
50. Mobile advertising spend is projected to account for 60.4% of all digital advertising spend by
2016 and 72.2% of all digital advertising spend by 2019. 51
The Ultimate Guide to DEM
Digital Experience Monitoring 8
NOT MEETING ONLINE USER EXPECTATIONS IS COSTLY
Lost Revenue
Every second of delay will lead to a 7% decrease in web conversions.52
Additionally, 75% of shoppers who experience a website that freezes,
crashes, is too slow, or involves a convoluted checkout process would
no longer buy from that site.3
Productivity Losses
Underperforming web-based internal systems severely hinder employee
productivity. 68% of managers surveyed in a recent study cited “slow
Internet” or “inability to access documents from a network” as affecting
their productivity.53
Brand Damage
Poor web experiences generate social media negativity and deter other
prospects, jeopardizing future revenue. In fact, 4% of unsatisfied customers
(such as those impacted by an outage or a slow loading page) will
complain.54 And just one negative review can cost you 30 customers.
Additional Marketing Costs
Inbound marketing leads will be lost when your site underperforms and
frustrated visitors leave without completing conversion goals. As a
result, you will need to increase your marketing spend to drive additional
web traffic to recoup lost leads.
The Ultimate Guide to DEM
Digital Experience Monitoring 9
MEETING AND EXCEEDING USER EXPECTATIONS DRIVES MORE BUSINESS!
Site speed affects your brand image—80% of people said online reviews and search results (Google favors quicker sites in ranking) determine what they think about a company, and social networks influence 49% of consumers.55
According to a recent study of more than 2,500 online consumers in the
US and UK, making each page in a transaction 2 seconds faster resulted
in more than double the number of completed transactions.4
Walmart optimized their site and found that every 100 ms of improvement
led to a 1% increase in revenue.56
Trulia decreased their page load time by 21%, resulting in an 11% increase
in sales.57
The Ultimate Guide to DEM
Digital Experience Monitoring 10
IDENTIFYING THE CAUSES OF POOR DIGITAL EXPERIENCES
Now that you know how delivering a poor digital experience can be a
detriment to your online success, the next step is understanding where the
biggest speed and availability risks are, and what measures you can take to
mitigate these risks.
The Ultimate Guide to DEM
Digital Experience Monitoring 11
RELIANCE ON THIRD-PARTY SERVICES
If an application moves from a single infrastructure with 99.99% availability to an open one relying on five different providers with 99.99% availability each, the result is an infrastructure with 99.95% availability.
Websites are a complex
ecosystem of social media,
private and public clouds, internal
components, content delivery
networks (CDNs), ads, reviews,
marketing, and analytics engines.
Just one underperforming
component within this delivery
chain can severely impact your
performance and could bring
business to a halt. Even worse,
site owners have little to no control over the performance of third-party
services. Furthermore, because of the proliferation of third-party services, a
seemingly isolated incident at the service provider may affect many more
companies than just themselves.
Third-party tag removed (errors stopped), then reinstalled
The Ultimate Guide to DEM
Digital Experience Monitoring 12
CHANGE AND HUMAN ERROR
In a new, agile development world, change is
happening constantly on the web, whether a
developer pushes new code or content, or an
infrastructure update is made. Due to
complexity or a lack of understanding of
performance impact, change can cause
unexpected degradations. Furthermore, the
web is coded and maintained by humans,
and humans make mistakes. Unfortunately,
these risks exist by nature and cannot always
be prevented.
NETWORK AND LOCATIONS
The speed of the web is limited by physics—
the distance of the end user to the server
dictates how long it takes to deliver the
webpage. The further the user from the server,
the higher the latency. Wireless, broadband
and fiber networks also all provide different
connection speeds, changing how fast a
webpage renders to the end user. While you
have no control over where your end users
browse from and which networks they use,
you can place your datacenters closer to them to mitigate factors outside of
your control.
Large gap in script loading due to coding error
Great performance in Phoenix….your smallest market
The Ultimate Guide to DEM
Digital Experience Monitoring 13
LOAD ON SERVERS, WEB TRAFFIC FLUCTUATIONS
Sometimes an event or promotion can drive a
burst of traffic to a website. If there are not
ample resources available on the server side
to handle the increased load, the server
performance will rapidly deteriorate until
reaching the point of failure. A website in a
public cloud environment can be affected by
resource-hungry neighbors in the same way.
WEBSITE FRONT-END CODE
How your website is coded and fetches
resources affects performance. As a rule, it is
best to make sure your CSS and JS are non-
blocking, the page makes as few requests as
possible, and all images are optimized for
performance (compressed and sprited). Keep
in mind that desktop and mobile browsers
render webpages differently.
Major delays caused by poorly configured JS
Servers can’t handle traffic spike from promotion
The Ultimate Guide to DEM
Digital Experience Monitoring 14
MEASURES TO REDUCE YOUR RISK
1. Before expected peaks of traffic, verify that your infrastructure (load
balancers, front-end servers, edge servers, databases) can handle the
increased load.
2. Solutions such as tag management systems and CDN load balancing can
reduce dangers stemming from the complex delivery of assets and
mitigate risk of third-party issues.
3. Understand from where and how the majority of your users access your
site and develop responsive design based on your data. If necessary,
leverage a CDN to serve content closer to your global end users.
4. There are many different free tools (such as Google PageSpeed and
Yahoo YSlow) that can crawl your page and score it based on best
practice guidelines. Be careful though—every website is unique, so a high
score does not necessarily guarantee a fast website.
5. If development resources are scarce for front-end performance
improvement, rely on front end optimization solutions to automatically
apply optimizations to the website.
6. Desktop and mobile browsers render webpages differently; make sure that
these variations are tested for and optimized during development. Beware of
potential negative performance impact when using jQuery on mobile.
The Ultimate Guide to DEM
Digital Experience Monitoring 15
DISCOVERING DIGITAL EXPERIENCE ISSUES
Knowing what can cause performance degradations during development is one
thing; spotting a performance issue out in the wild (that may impact users) is
another. There are multiple methods for keeping an eye on the health of your
digital interactions, but the success of each depends on your specific needs.
5 DEM STRATEGIES
Many organizations rely on a combination of approaches to oversee their
digital experiences, including:
1. Help desk, tech support, and social media — responsive monitoring
2. Real user measurement (RUM) — passive monitoring for end-user experience
3. Application monitoring — passive monitoring for applications and
infrastructure
4. Cloud-based synthetic monitoring — active monitoring
5. On-premise synthetic monitoring — active monitoring
The Ultimate Guide to DEM
Digital Experience Monitoring 16
HELP DESK, TECH SUPPORT, AND SOCIAL MEDIA
Monitoring social media feeds and listening to your customers is an excellent way to
understand their experience. A timely, human response to customers experiencing issues
strengthens your business credibility and brand loyalty. While handling issues from calls to
support or posts on social media means you are being responsive, it also means an issue
has already affected end users.
BENEFITS LIMITATIONS
No added cost with support and help desk
systems already existing.
Direct interaction with customers.
Ability to discover performance edge cases on
the less common navigational paths (irregular
user onsite actions).
Issues have already impacted the end user and
your revenue.
Customers are not performance experts or
technical enough to report exactly what’s going
on.
Not everyone will contact you immediately (if at
all) when they are having problems, so issues
may be missed or caught late.
If the site is down or unusable, a customer may
not know how to contact you.
The Ultimate Guide to DEM
Digital Experience Monitoring 17
REAL USER MEASUREMENT (RUM)
Real user measurement (RUM) tracks web performance from the browser using
JavaScript embedded on the webpage. The solution measures the performance of the
pages as experienced by the end users. Therefore, the performance data collected is
impacted not only by your infrastructure, third parties, and CDNs, but it includes factors
outside your control like the end user’s device, Internet connection, and others.
BENEFITS LIMITATIONS
Measures the webpage’s performance as
experienced by the end users.
Measures the performance of all pages
accessed by end users.
Gives a picture of user experience across users,
geographies, and devices.
Determines if code changes or
architectural/infrastructure changes had the
desired efforts or cause errors and/or
performance degradations.
Correlates performance data to user
engagement and/or revenue metrics.
Cannot detect or measure downtime or where it
occurred (DNS, TCP Connection, Server, etc.)
Can miss performance problems during light or
no traffic time periods.
Cannot benchmark website performance
against competition.
No screenshots, filmstrips, Ping, Traceroute.
Performance numbers driven by users in
particular geographies and time zones
Navigation Timing not supported in all
browsers.
Geo-IP data is not 100% accurate.
The Ultimate Guide to DEM
Digital Experience Monitoring 18
APPLICATION MONITORING
Application monitoring allows you to monitor the health, availability, and performance of
your applications and underlying infrastructure from within the datacenter. Application
Monitoring is done primarily via deployed agents but there are different collection
mechanisms, including Java byte code manipulation techniques, real-time transaction
tracing, HTTP appliances, network packet sniffers, and others.
BENEFITS LIMITATIONS
Provides deep visibility into application
performance down to a specific fault line of
code, which aids with triage and
troubleshooting.
Gives a complete understanding of internal
infrastructure performance and resource
utilization; a must for proper capacity planning.
Lack of visibility on the performance
experienced by end users.
Lack of visibility into third-party components
and services.
Cannot benchmark your web performance with
competition.
Some application monitors are intrusive and
not suited for production environments.
Very costly pricing model for enterprises with
hundreds of servers.
The Ultimate Guide to DEM
Digital Experience Monitoring 19
CLOUD-BASED SYNTHETIC MONITORING
Cloud-based synthetic web monitoring proactively monitors your website 24/7/365 from
specific geographies and ISPs. It relies on software-based agents (backbone, last mile,
wireless and private nodes) distributed throughout the world in a “clean lab” setting to
simulate user experience. Synthetic testing measures and validates key business
processes and functions in your website (shopping carts, CRM record retrieval, web lead
registrations, conversion goals, etc.). Synthetic agents alert on availability and
performance issues at the first sign of trouble.
BENEFITS LIMITATIONS
Continuously monitors your website from a set
number of locations, 24/7, even when no users
are on the site.
Reduces external noise by testing from
backbone ISPs.
Lab environment can be used for SLA
management and verification.
Detects early performance issue signs.
Provides full insight on all requests.
Captures screenshots and headers for further
troubleshooting.
Benchmarks performance against competition.
Lack of visibility into the experience of actual
end users visiting the site.
Need to correlate performance with
engagement and revenue metrics from RUM or
Web Analytics tools.
Can miss performance problems for less
visited webpages, not considered on your
monitoring plans.
Limited geography and ISP locations.
May become costly to monitor every webpage
and navigation path.
The Ultimate Guide to DEM
Digital Experience Monitoring 20
ON-PREMISE SYNTHETIC MONITORING
On-premise synthetic web monitoring proactively monitors your internal applications
24/7/365 from your own locations. This can be from where your applications are hosted,
such as with first-mile monitoring of SaaS applications, are from the branch offices, call
centers, or any other locations your internal users access your applications. It relies on
hardware devices, servers or virtual machines deployed inside your firewall to test
network-connected applications for key business processes and functions. Examples of
the kinds of applications monitored in this way include SaaS applications for your internal
users, point-of-sale and inventory applications in a retail store, or call center applications in
customer service or tech support operations. These synthetic agents alert on availability
and performance issues at the first sign of trouble.
BENEFITS LIMITATIONS
Continuously monitors your applications from
your internal locations, 24/7, even when no
users are accessing the applications
Monitoring nodes can go wherever your
applications and services go
Eliminates external noise by testing from
internal networks
Can be used for SLA management and
verification of internally-consumed SaaS
applications and cloud services
Detects early signs of performance issues
Provides full insights on all service requests
May not always reflect experience of actual
end-users depending on environment variables
Need to correlate performance with
engagement and revenue metrics from RUM or
analytics tools.
Can miss performance problems for
applications and workflows that are less
frequently used
Monitoring limited to your own physical
locations.
Lack of visibility into external network
performance and Internet services
The Ultimate Guide to DEM
Digital Experience Monitoring 21
CHOOSING THE RIGHT MONITORING STRATEGY
All companies should monitor their social feeds; being responsive,
informative, and timely is the best way to deal with unhappy customers
experiencing performance issues. That said, catching and resolving errors
before your users take to social with verbal pitchforks requires a more
comprehensive monitoring strategy. We’ve laid out the various monitoring
options (RUM, application, and synthetic); how do you decide which ones to
use?
WHICH TOOL IS RIGHT FOR ME?
While RUM is beneficial to get broad-spectrum metrics and real user data,
inconsistencies between users and user groups can skew data. For example,
there can be many permutations between user environments (location,
device, OS, browser, connection speed, etc.) and too small of a population
size per environment to draw significant conclusions from the data. On the
other hand, too large of a user group can cause the holistic data to be
weighted toward that population.
Application Monitoring is very helpful in detecting, debugging, and
troubleshooting the root cause of the problem. However, it can only detect
issues within your web infrastructure, and cannot detect issues with DNS,
ISP Providers, CDNs, and third parties.
The Ultimate Guide to DEM
Digital Experience Monitoring 22
The biggest issue with RUM and application monitoring, or any passive
monitoring solution, is that you cannot determine if there is an outage
outside of the application/website layer (where the data is being collected).
Additionally, if there is no traffic to the site or if a user cannot interact with
the page, then passive monitoring solutions do not collect any data, so there
is a chance that you miss substantial performance issues. Always remember
that any performance issues you detect with these solutions means the
problem has already impacted the end user.
For a complete approach to web performance and user experience optimization, using application, RUM, and synthetic is highly recommended.
Application monitoring lets you troubleshoot and debug issues at the code
level when they occur within your infrastructure. RUM gives you the real-time
user insight needed to improve the user experience from all locations, and on
most devices and browsers. Synthetic monitoring controls the variables that
impact the experience of individual users. This allows you to run calibrated
tests and collect performance data from any location (within or outside a
firewall) to run analysis, spot trends, benchmark against competitors and
detect early signs of performance degradations before users are impacted.
The Ultimate Guide to DEM
Digital Experience Monitoring 23
WHY YOU MUST HAVE SYNTHETIC MONITORING IN YOUR MIX
You might be using application monitoring and RUM and think that is
sufficient monitoring for your website or web application. However, when you
explore the benefits of synthetic monitoring, you’ll understand why it needs
to be a part of your monitoring strategy.
Additional benefits of synthetic monitoring include:
1. Enhanced Brand Protection
Be the first to know there is a problem so you can react and resolve issues faster (reroute
traffic, temporarily remove widgets, give updates on social media, etc.)
2. Lower Costs and Boosted Internal Productivity
Spend less time troubleshooting and dealing with frustrated users. Identify early
performance/availability degradations, before the call center gets flooded.
3. SLA Compliance
Hold your technology partners to their SLAs and ensure you are meeting internal goals.
4. Discovery and Measurement of Optimization Opportunities
Find new areas for web optimization to increase user satisfaction and conversion rates.
Test and deploy new technology with confidence they will have an impact on end-user
experience.
5. Competitive Intelligence
Benchmark competitors and industry peers to define market-leading performance goals.
The Ultimate Guide to DEM
Digital Experience Monitoring 24
MONITORING CHECKLIST TO GET YOU GOING
1. Identify key business functions and mission-critical transactions to test and
monitor
Examples: Web registration forms, shopping carts, database record retrieval,
CDNs, DNS, etc.
2. Identify worldwide monitoring locations
You should take key factors such as which countries your customers are currently
coming from, upcoming marketing campaigns or new geo-targeted expansion
plans into consideration.
3. Decide which websites and networks you will test
Use Google or other web analytics reports to baseline your current users, so you can
verify your website performs properly for top revenue generating customers.
4. Identify the “weak links” on a webpage—what makes it unusable?
Examples: Broken images, ads not displaying, broken checkout process, hanging
page, hard failure, etc.
5. Define monitoring frequency and alerting policies
Decide what type of problems will trigger an alert (internal components only,
specific hosts, third-party violations, etc.) and assign ownership and who will
receive which alerts. Test crucial functions more frequently.
Unified Synthetic & RUM Data
This guide provides an adaptable framework based on industry best practices that will help IT Operations and Development leaders develop a request for information when considering a synthetic and RUM solution. By developing an optimal monitoring strategy, your organization will improve IT operational e�ciency, increase revenue, and strengthen your brand.
Capabilities
Web (browser and transaction) tests types only requiring multi-vendor approach
for more monitors (e.g., network)
Synthetic agents on AWS cloud or limited coverage in shared data centers,
consumer ISPs, and on last mile that are unreliable and inaccurate
Real user measurement data captured via separate tool
Collects large amount of data, requiring additional analysis to identify issues
Leading Solution IT Benefits
Digital Experience Monitoring Selection Guide
Diverse test types for visibility into application, network, and infrastructure
layers from the end user’s perspective on one platform
Decreased risk of micro-outages and more information to troubleshoot
Fewer tools, consolidated toolset
Improved visibility across services that were not monitored before (DNS)
Automatically raises issues to the surface with little e�ort from analyst to
make data useful
Synthetic agents on key networks, consumer ISPs, mobile, and last mile DSL
connection, and On-Premise option
Real user measurement data captured in context of synthetic data
Decreased risk of micro-outages and improved MTTR
Increased visibility into overall customer experience and impact of
network issues, peering relationships
Separate systems capturing time-series data
Need multi-vendor approach, including an ITOA tool to upload and analyze both
types of data
One system capturing time-series data
Dashboards and analysis can support and overlay both data types in one UI
Improved user experience visibility
Stronger reporting and analysis
Improved workflow and stronger correlation
Clunky user interface and/or multiple portals to log in to
Cumbersome scripting capabilities, requiring days to script and upload to platform
Single UI with convenient navigation, simple and e�cient workflows
Create, upload, and edit tests in real time with Selenium scripting
Accelerated user adoption
Improved MTTR
Improved user experience visibility
More e�cient workflow
Platform built on SQL database that processes data in batches, can’t handle
large data sets, can’t easily integrate data from other tools, and requires a
full-time database administrator
Limited data retention: one month or less of record-level data, and less than
one year of summary data (one single data point per day)
Platform built on NoSQL database that delivers data instantly to user, handles
large data sets better, and does not require a full-time database administrator
Longer data retention: Over three months of record-level data on all tests
results and three years of un-aggregated summary data
Improved workflow
Reduced mean time to detect and identify, resulting in lower MTTR
Stronger reporting and analysis
No need to keep an in-house data warehouse
Generic dashboards with limited/no option to customize by user groups
Pre-aggregated reports with limited options to personalize detailed reports
No dimensional analysis across array of chart types with minimal non-flexible
visualizations; limited to summary views
User-defined and public dashboards to send to non-users, performance
threshold dashboards tailored for NOC use
Scheduled and chartable reports customized by user and division
Flexible multi-dimensional analysis, historical comparisons for baseline and
benchmarking, statistical analysis, and charting
Decreased data complexity
Stakeholders have insight to make strategic decisions
Ability to present customized data to di�erent stakeholders
Basic alerts via email with minimal/no threshold option and troubleshooting
information not readily available
Limited/zero troubleshooting information collected, prompting end user to run
instant tests to identify issue
Personalized alerts by custom thresholds and baseline deviation delivered with
contextual troubleshooting information
Intelligent triggers for extended debug information on error (DNS traversal,
network traceroute, and ping)
Faster understanding of problem
Improved MTTR
Decreased FTE Costs
Summary-level data for HTTP with limited ability to drill down
Need multi-vendor approach to get network and server-level visibility
Record-level data visibility with drill down into waterfall data components for
application and network
Retrieve server name information for backend visibility
Reduced mean time to detect and identify, resulting in lower MTTR
Decreased FTE costs
Data Speed & Quality
Vantage Points
Synthetic Test Types
Real User
Data Deep-Dive
Data Speed & Quality
Problem Identification
User Interface
Dashboards, Reporting, &
Analysis
Traditional Monitoring Solution
The Ultimate Guide to DEM
Digital Experience Monitoring 26
REFERENCES 1 http://www.fastcompany.com/1825005/how-one-second-could-cost-amazon-16-billion-sales
2 https://doubleclick-publishers.googleblog.com/2016/09/the-need-for-mobile-speed.html 3 https://doubleclick-publishers.googleblog.com/2016/09/the-need-for-mobile-speed.html 4 https://doubleclick-publishers.googleblog.com/2016/09/the-need-for-mobile-speed.html 5 https://doubleclick-publishers.googleblog.com/2016/09/the-need-for-mobile-speed.html 6 http://sixrevisions.com/mobile/pay-attention-to-mobile-web-performance/ 7 https://www.thinkwithgoogle.com/nordics/research-study/the-need-for-mobile-speed-how-mobile-latency-impacts-publisher-revenue/ 8 https://doubleclick-publishers.googleblog.com/2016/09/the-need-for-mobile-speed.html 9 https://doubleclick-publishers.googleblog.com/2016/09/the-need-for-mobile-speed.html 10 https://doubleclick-publishers.googleblog.com/2016/09/the-need-for-mobile-speed.html 11 https://www.soasta.com/blog/23-stats-mobile-web-performance-monitoring/ 12 https://www.soasta.com/blog/23-stats-mobile-web-performance-monitoring/ 13 https://www.soasta.com/blog/23-stats-mobile-web-performance-monitoring/ 14 https://www.akamai.com/es/es/multimedia/documents/state-of-the-internet/akamai-state-of-the-internet-report-q1-2016.pdf 15 https://www.soasta.com/blog/23-stats-mobile-web-performance-monitoring/ 16 https://www.soasta.com/blog/23-stats-mobile-web-performance-monitoring/ 17 http://www.globalwebindex.net/hubfs/Reports/GWI_Device_Report_-_Q3_2015_Summary.pdf 18 http://searchengineland.com/its-official-google-says-more-searches-now-on-mobile-than-on-desktop-220369
19 http://heidicohen.com/2015-mobile-marketing/ 20 http://heidicohen.com/2015-mobile-marketing/ 21 https://www.soasta.com/blog/23-stats-mobile-web-performance-monitoring/ 22 https://techcrunch.com/2015/06/02/6-1b-smartphone-users-globally-by-2020-overtaking-basic-fixed-phone-subscriptions/ 23 http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/mobile-white-paper-c11-520862.html 24 https://hostingfacts.com/internet-facts-stats-2016/ 25 https://blog.kissmetrics.com/loading-time/ 26 https://www.doubleclickbygoogle.com/articles/mobile-speed-matters/ 27 http://blog.accessdevelopment.com/customer-loyalty-statistics-2015-edition 28 https://www.soasta.com/blog/23-stats-mobile-web-performance-monitoring/ 29 https://blog.kissmetrics.com/loading-time/
30 https://mobiforge.com/research-analysis/how-important-is-web-performance 31 https://www.fastcompany.com/1825005/how-one-second-could-cost-amazon-16-billion-sales 32 https://www.thinkwithgoogle.com/intl/en-gb/research-studies/what-users-want-most-from-mobile-sites-today.html 33 http://blog.accessdevelopment.com/index.php/2013/11/the-ultimate-collection-of-loyalty-statistics 34 https://www.radware.com/newsevents/pressreleases/radware-reveals-76-percent-travel-websites-arent-optimized/
The Ultimate Guide to DEM
Digital Experience Monitoring 27
35 https://hostingfacts.com/internet-facts-stats-2016/ 36 http://www.emarketer.com/Article/Mcommerces-Rapid-Growth-Primarily-Coming-Smartphones/1013909 37 http://www2.deloitte.com/us/en/pages/about-deloitte/articles/press-releases/digital-influences-in-retail-store-sales.html 38 http://www2.deloitte.com/content/dam/Deloitte/us/Documents/consumer-business/us-retail-mobile-influence-factor-062712.pdf 39http://newsroom.bankofamerica.com/files/doc_library/additional/2015_BAC_Trends_in_Consumer_Mobility_Report.pdf 40 http://www2.deloitte.com/us/en/pages/about-deloitte/articles/press-releases/retail-digital-divide.html 41 https://hostingfacts.com/internet-facts-stats-2016/ 42 https://think.storage.googleapis.com/docs/micromoments-guide-to-winning-shift-to-mobile-download.pdf 43 http://www.criteo.com/media/1894/criteo-state-of-mobile-commerce-q1-2015-ppt.pdf 44 https://skift.com/2015/07/21/the-rise-of-the-mobile-channel-changing-booking-patterns-and-business-models/ 45 http://www.emarketer.com/Article/By-2016-Most-Digital-Travel-Bookers-Will-Use-Mobile-Devices/1013248 46 http://blogs.forrester.com/thomas_husson/15-11-10-how_mobile_will_transform_business_in_2016 47 https://www.internetretailer.com/2014/10/01/mobile-shoppers-convert-160-more-often-optimized-sites 48 http://sloanreview.mit.edu/projects/embracing-digital-technology/ 49 http://heidicohen.com/2016-mobile-marketing-trends/ 50 http://www.emarketer.com/Article/Mobile-Will-Account-72-of-US-Digital-Ad-Spend-by-2019/1012258 51 http://www.emarketer.com/Article/Mobile-Will-Account-72-of-US-Digital-Ad-Spend-by-2019/1012258
52 http://www.aberdeen.com/aberdeen-library/5136/RA-performance-web-application.aspx 53 http://alliedtelesis.co.uk/p-5190.html 54 http://returnonbehavior.com/2010/10/50-facts-about-customer-experience-for-2011/ 55 http://blog.kissmetrics.com/speed-is-a-killer/ 56 http://minus.com/msM8y8nyh/2e 57 http://www.stubbornella.org/content/category/general/geek/performance-geek-general/
top related