introducing mobile testing to your organization
TRANSCRIPT
T16 Mobile Testing
5/2/2013 1:30:00 PM
Introducing Mobile Testing to Your
Organization
Presented by:
Eric Montgomery
Progressive Insurance
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
Eric Montgomery
A system test engineer for Progressive Insurance, Eric Montgomery joined Progressive as a C/C++/C# developer, programming their quoting, rating, and renewal logic. On a project to replace Progressive's mainframe servicing system with a .NET version, Eric transitioned to testing, first developing headless test harnesses for business logic functions and then becoming test lead of one of the largest test events ever run at Progressive. Since 2011, Eric has been the test lead for Progressive’s mobile transformation program, which created several native apps, including the award-winning Progressive flagship app and the “mobilization†of many of Progressive's web sites.
4/19/2013
1
Introducing Mobile Testing
to Your Organization
Eric W Montgomery
System Test Engineer | Progressive Insurance
Who is Progressive?
» Mayfield Village, OH
» Offering insurance since 1937
» Web awards (Webby)
» Mobile awards (Keynote Brand Innovation)
4/19/2013
2
Who Am I?
» Cleveland native
» Miami University, Oxford, OH
» Progressive since 2003
» Mobile Test Lead, 2011-2013
How it all Began
» 2 developers (1 iOS/iPhone, 1 Android)
» 2 devices (1 iPhone, 1 Android)
» 1 tester
» Mobile Program
4/19/2013
3
» People
» Devices, tools
» Testing techniques
» Stories, experiences
Agenda
Overcoming the Fear of Mobile
» Common issue
» Practice testing mobile with a device
» Discuss terminology
– App types: native, web, hybrid
– Device features
– Operating systems: Android, iOS
– so many more
4/19/2013
4
People
» Key component
» Be willing to adjust quickly
» Constant
learning
Mobile Testers’ Forum
» Meets bi-weekly
» Variety of topics
– Project work
– Technology trends
– Testing techniques
» Network
» Answer questions without
asking
» Presentations
» Learn/teach technologies
4/19/2013
5
Devices
» 2 devices (1 iPhone, 1 Android), supplement with
emulators
» How we learned what phone our CEO had
» Emulators are insufficient for System Testing
– Not exactly the same
– Not handling a device
– Different hardware: camera,
location services
– Android: manufacturers add code
» Still good for unit, smoke testing
Device Selection
MOTXT912B, 8.98%
SPH-L710, 5.50%
GT-I9300, 4.99%
SGH-I747, 4.39%
SPH-D710, 4.06%
SGH-T999, 3.80%
SGH-989, 3.26%
XT875, 2.27%
SGN, 2.18%
PC36100, 2.13%
SGH-I727, 1.89%
HDH, 1.82%
ADR6400L, 1.57%
MDX2, 1.51%
X325C, 1.49%
SCH-I500, 1.48%
MDX2, 1.47%PJ83100, 1.47%
ADR6350, 1.44%
SCH-I535, 1.41%
SGH-I717, 1.41%
X515, 1.37% SCH-I510, 1.35% XT894, 1.27%
XT907, 1.26% SGH-I997, 1.24%
SGH-I777, 1.01%
PH39100, 0.94%
ADR6425LVW, 0.94%
SCH-I405, 0.92%
GT-I9000, 0.90%
XT862, 0.87%
VILLE, 0.87%
Z710, 0.85%
GLAC, 0.84%
SGH-T769, 0.83%
SPH-D700, 0.80%VS840, 0.80%
MB855, 0.79%
SGH-T679, 0.74%
MB886, 0.74%MS770, 0.72% N860, 0.70%MB865, 0.57%
P999, 0.55%SPH-930, 0.55%MD2, 0.54%
GNEX, 0.51%
VS920, 0.50%
ADR6300, 0.48%
C771, 0.47%
P9070, 0.47%AMZ, 0.47%ELC, 0.46%SCH-I200, 0.45%MB860, 0.41%VS910, 0.41%
SCH-R760, 0.39%E739BK, 0.38%MYTSLD, 0.37%SGH-I897, 0.37%PH44100, 0.36%C800, 0.35%ADR6330, 0.34%G2TCH, 0.33%MS910, 0.32%
SCH-R720, 0.30%ADR6410LVW, 0.29%
SCH-R930, 0.28%P925, 0.27%
SPH-D600, 0.26%M886, 0.25%LS840, 0.25%
SGH-T959, 0.24%SCH-R920, 0.23%SCH-T839, 0.23%SGH-I577, 0.23%
Other, 8.70%
4/19/2013
6
Device Selection
» Consider project goals
» Ask questions
» Eliminate options!
» Equivalence classes
» Crowdsourcing
A47%
B25%
C10%
D10%
Other8%
Device Selection
4/19/2013
7
Device Selection: Using Data
» Data
» Market trends
» Making decisions jointly with customer
Devices: Owning
» Warehousing/library
» Maintaining
– Cleaning
– Operating systems
» Cords
4/19/2013
8
Testing Tools, Tips & Techniques
Old Techniques Made New Again
» Error handling
– Edits
– Nonsense messages
– Error logging
» Dates
– Valid ranges
– Formats
Testing Tools, Tips & Techniques
Automation
» How it works
» Native
– Tree structure
– Unit/Integration level tests
» Web
4/19/2013
9
Testing Tools, Tips & Techniques
New Techniques
» Brain
– Perspective
– Pair testing
» Devices
» Chrome
– Developer panel
– Remote debugging
» View source bookmarklet for iOS
» Screen shots
– Recording apps
– Camera, video
» Bug reports/exploratory sessions
Testing Tools, Tips & Techniques
New Techniques
» Settings
– User settings/airplane mode
– Moving between networks
– Cookies, JavaScript
» Navigation
– Service calls
– Links
» Keyboards
– Numeric
– iOS vs. Android
» Data Usage
4/19/2013
10
Testing Tools, Tips & Techniques
New Techniques
» Mobile
– Rotations
– Location services, camera
– Moving around
– Interruptions
» Look and Feel/UX
– Consistency
– Font, colors, headings, images
– Spelling, grammar
– Alignment, margins
» Change browsers, devices
» Deployment
Summary
