the essentials of mobile app performance testing and monitoring

The Essentials of Mobile App Performance Testing and Monitoring

April 26, 2012

1. The Mobile Web Today2. Mobile Performance Testing3. Mobile App Monitoring4. Summary/Q&A

Agenda

Housekeeping

• Presentation will last 30 minutes• Submit questions via the chat window• Slides will be made available tomorrow

Featured Speakers

Stanton ChampionProduct Marketing ManageruTest

Frank DaysVP, MarketingCorrelsense

THE MOBILE WEB TODAY…

1.2 billion mobile web users world wide

-IDG Feb

2012

74% of users will bounce after waiting 5 seconds for a mobile site to load

Gartner predicts that by 2014, 90% of all corporations will support applications on personal devices – Feb 2012

Mobile’s Just Getting Started

0.5

1.0

1.5

2.0

2.5

Units$MM

2000 2016Year

Tablets

Smartphones

Personal Computers

So

Lo MoSocial Local Mobile

Web Testing Matrix Is Tough

OS & Browser

Features

Test Coverage

Mobile Testing Matrix Is Insane

OS & Browser

Features

Handset Makers& Models

Test Coverage

Mobile Testing Matrix Is Insane

OS & Browser

Features

Handset Makers& Models

WirelessCarriers

Test Coverage

Mobile Testing Matrix Is Insane

OS & Browser

Features

Handset Makers& Models

WirelessCarriers

Location

Test Coverage

Increasingly Complex Testing• The laughably simplistic notion of a “test matrix”

Not Just Testing, but More Complex Monitoring from Test to Production

Wider range of devices are connected to a more complex IT world

• Challenges

– Complaints performance

– No visibility into mobile app

transactions

– Limited/no load testing prior

to mobile rollout

– Huge headache for CIO

• Opportunities

– Mobile technologies to

Improve Productivity

– For “mobile workforce”

– Real-time data access

• Examples:

– CRM

– Order entry

– ERP

Both Are Needed to Improve End User Experience

ESSENTIALS FOR PERFORMANCE TESTING

Agile vs.Waterfall

Manual vs. Automation

Managing vs.Leading Teams

Exploratory vs.Test Cases

Testers vs. Engineers

In-House vs. Outsourced

Offshore vs.Nearshore

In T

he L

ab

Lab-Based Testing Innovation

In T

he L

ab

In

TheWild

Users Work & Live 'In The Wild'

• Recognize it will never again be sufficient on its own• Build your company’s plan for testing ‘In The Wild’

– Real-world conditions: Live testers, real devices, imperfect connectivity

– Mirror your user base• Technologically: OS, browser, anti-virus, device, carrier• Geographically: Continent, country, city, language• Demographically: Age, gender, education, employment, industry

Look Beyond The Lab

Your Connection to the Cloud

Big Fat Pipe

Your Automated Load Testing Tool’s Connection to the Cloud

Big Fat Pipe

Your Users’ Connections to the Cloud

Testing 'In The Wild' Finds New Bottlenecks

• Different users have different connection pathways– Mobile towers

– Network providers

– Backbone connections

– Content Delivery Networks

– DNS providers

• Above can add latency, making the user’s experience worse

'In The Wild' testing helps you identity down-stream network problems that you can’t find in the lab

Smarter Load Testing With Experts

• People don't load test often, few are experts • Professional testers are valuable• Technical understanding improves perception of

problems and root causes• When testing 'In The Wild', need professionals

with skills and experience

Coupled with end-to-end monitoring

• You need smart, adaptable monitoring that can work in the wild and inside your data center

ESSENTIALS FOR MOBILE APP MONITORING

1. Be Strategic

You need

measurable SLAs

To tie mobile

strategy to biz

goals

2. Be Transparent

Keep biz stakeholders in the loop and ensure SLA data is accessible

3. Be Paranoid

Datacenter

WEB

MQ/ESB

DCOM

CORBA

SERVER

APP SVR

PROXY

LDAP

WebServices

Watch Every Transaction for Every User

Total End-User Response TimeBrowser Rendering Time

Track Key User Experience Metrics

Rend

erin

g

Mobile Real User Monitoring

Datacenter TimeNetwork Latency

Mobile

4. Be Smart - Understand what is happening across the Space-Time continuum

Look Across Time

Look Across Locations

Know When There is a Problem

Summary

• When testing mobile

– Test outside the lab

– 'In The Wild'

– Use professionals

• When monitoring mobile

– Be strategic

– Be transparent

– Be paranoid

– Be smart

• Mobile web is complex

• New testing and monitoring tools

are needed

• End user experiences are still

critical

Questions

Get your free copy SharePath RUM!

http://www.real-user-monitoring.com/

Contact uTest:http://www.utest.com/Phone: 1-800-445-3914Follow us:

twitter.com/uTest

facebook.com/uTest

Contact Correlsense:www.correlsense.com/demowww.real-user-monitoring.cominfo@correlsense.comFollow us:

twitter.com/correlsense

facebook.com/correlsense

Contact Us At http://www.utest.com/contact-

customers GET STARTED

TODAY!