The Essentials of Mobile App Performance Testing and Monitoring
April 26, 2012
1. The Mobile Web Today2. Mobile Performance Testing3. Mobile App Monitoring4. Summary/Q&A
Agenda
Housekeeping
• Presentation will last 30 minutes• Submit questions via the chat window• Slides will be made available tomorrow
Featured Speakers
Stanton ChampionProduct Marketing ManageruTest
Frank DaysVP, MarketingCorrelsense
THE MOBILE WEB TODAY…
1.2 billion mobile web users world wide
-IDG Feb
2012
74% of users will bounce after waiting 5 seconds for a mobile site to load
Gartner predicts that by 2014, 90% of all corporations will support applications on personal devices – Feb 2012
Mobile’s Just Getting Started
0.5
1.0
1.5
2.0
2.5
Units$MM
2000 2016Year
Tablets
Smartphones
Personal Computers
So
Lo MoSocial Local Mobile
Web Testing Matrix Is Tough
OS & Browser
Features
Test Coverage
Mobile Testing Matrix Is Insane
OS & Browser
Features
Handset Makers& Models
Test Coverage
Mobile Testing Matrix Is Insane
OS & Browser
Features
Handset Makers& Models
WirelessCarriers
Test Coverage
Mobile Testing Matrix Is Insane
OS & Browser
Features
Handset Makers& Models
WirelessCarriers
Location
Test Coverage
Increasingly Complex Testing• The laughably simplistic notion of a “test matrix”
Not Just Testing, but More Complex Monitoring from Test to Production
Wider range of devices are connected to a more complex IT world
• Challenges
– Complaints performance
– No visibility into mobile app
transactions
– Limited/no load testing prior
to mobile rollout
– Huge headache for CIO
• Opportunities
– Mobile technologies to
Improve Productivity
– For “mobile workforce”
– Real-time data access
• Examples:
– CRM
– Order entry
– ERP
Both Are Needed to Improve End User Experience
ESSENTIALS FOR PERFORMANCE TESTING
Agile vs.Waterfall
Manual vs. Automation
Managing vs.Leading Teams
Exploratory vs.Test Cases
Testers vs. Engineers
In-House vs. Outsourced
Offshore vs.Nearshore
In T
he L
ab
Lab-Based Testing Innovation
In T
he L
ab
In
TheWild
Users Work & Live 'In The Wild'
• Recognize it will never again be sufficient on its own• Build your company’s plan for testing ‘In The Wild’
– Real-world conditions: Live testers, real devices, imperfect connectivity
– Mirror your user base• Technologically: OS, browser, anti-virus, device, carrier• Geographically: Continent, country, city, language• Demographically: Age, gender, education, employment, industry
Look Beyond The Lab
Your Connection to the Cloud
Big Fat Pipe
Your Automated Load Testing Tool’s Connection to the Cloud
Big Fat Pipe
Your Users’ Connections to the Cloud
Testing 'In The Wild' Finds New Bottlenecks
• Different users have different connection pathways– Mobile towers
– Network providers
– Backbone connections
– Content Delivery Networks
– DNS providers
• Above can add latency, making the user’s experience worse
'In The Wild' testing helps you identity down-stream network problems that you can’t find in the lab
Smarter Load Testing With Experts
• People don't load test often, few are experts • Professional testers are valuable• Technical understanding improves perception of
problems and root causes• When testing 'In The Wild', need professionals
with skills and experience
Coupled with end-to-end monitoring
• You need smart, adaptable monitoring that can work in the wild and inside your data center
ESSENTIALS FOR MOBILE APP MONITORING
1. Be Strategic
You need
measurable SLAs
To tie mobile
strategy to biz
goals
2. Be Transparent
Keep biz stakeholders in the loop and ensure SLA data is accessible
3. Be Paranoid
Datacenter
WEB
MQ/ESB
DCOM
CORBA
SERVER
APP SVR
PROXY
LDAP
WebServices
Watch Every Transaction for Every User
Total End-User Response TimeBrowser Rendering Time
Track Key User Experience Metrics
Rend
erin
g
Mobile Real User Monitoring
Datacenter TimeNetwork Latency
Mobile
4. Be Smart - Understand what is happening across the Space-Time continuum
Look Across Time
Look Across Locations
Know When There is a Problem
Summary
• When testing mobile
– Test outside the lab
– 'In The Wild'
– Use professionals
• When monitoring mobile
– Be strategic
– Be transparent
– Be paranoid
– Be smart
• Mobile web is complex
• New testing and monitoring tools
are needed
• End user experiences are still
critical
Questions
Get your free copy SharePath RUM!
http://www.real-user-monitoring.com/
Contact uTest:http://www.utest.com/Phone: 1-800-445-3914Follow us:
twitter.com/uTest
facebook.com/uTest
Contact Correlsense:www.correlsense.com/[email protected] us:
twitter.com/correlsense
facebook.com/correlsense
Contact Us At http://www.utest.com/contact-
customers GET STARTED
TODAY!
