6 ways to speed up app testing
TRANSCRIPT
6 Ways to Speed Up App Testing
Presenter
Christopher P. WillisCMO – Perfecto Mobile
Over 18 years of experience growing companies in the technology sector.
A mobile industry pioneer, Chris helped set the direction for the marketing of mobile applications and mobile application development platforms over a decade ago.
Setting the bar for market success
Introducing Perfecto Mobile
Perfecto Mobile at a glance
Industry standout & rising star
FROST AND SULLIVAN CUSTOMER VALUE LEADERSHIP AWARD –MOBILE APPLICATION TESTING, GLOBAL, 2013
13Data Centers
50KTesting Hours / Month
99.5%Uptime
Global Coverage
Perfecto Mobile helps dev and test teams deliver continuous quality, accelerating the velocity of app delivery without compromises.
Demographics
350 Mobile Developers, Testers and QA Managers
Testing / QA71 %
Development 20 %Head of Mobile 9 %
Roles
1 – 5 apps48 %
6 – 10 apps11 %
11 – 20 apps16 %
20+ apps25 %
Development Scope
Current Process
Agile ProcessesWaterfall
86 %14 %
94 %
Now
Goal
WHAT TESTING FACTORS SLOW DOWN YOUR RELEASE CYCLES?
Question:
What testing factors slow down your release cycles?
?Rank what causes
delays during mobile app testing
& release
1. Manual testing takes too long.
2. Timely access to mobile devices.
3. Delays in device lab readiness.
4. Ineffective test automation.
5. Slow QA to Development feedback.
6. Too many defects declared non-
reproducible.
TOO MANY DEFECTS ARE DECLARED NON-REPRODUCIBLE
Obstacle #6
Too many defects are non-reproducible
If it’s hard to reproduce defects due to lack of evidence, it is hard to provide sufficient feedback to Dev teams.
Example: An international telecoms customer• Problem: Dev teams had difficultly replicating bugs
that offsite testing teams reported
• Solution 1: Video reporting that recorded entire test transactions was made sharable between Dev and Teams to close this loop
• Solution 2: Replicate the defect on the same device on which it was uncovered
EVIDENCE IS KEYSolution
TOO MANY DEFECTS ARE DECLARED NON-REPRODUCIBLE
Obstacle
• Video recordings shed light on bugs• Sharable video bridges Dev and Test teams during feedback
cycles, and dramatically improves velocity• Real devices, in the cloud, allow testers and developers to
collaborate over the same device, under the same user conditions
SLOW QA TO DEV. FEEDBACKObstacle #5
Slow QA to Developer FeedbackFifth factor, but slows down the cycle more than other
factors.
Example: Travel Site• Problem: Monthly release cycle, with 3 weeks of
development and 1 week for testing and bug fixing. As much work needed to happen in this 1 week as the previous 3.
• Solution: Embedded quality and testing throughout the development cycle. Unit tests make QA feedback visible and developers know where there code stands at all times.
CONTINUOUS QUALITYSolution
SLOW QA TO DEVELOPER FEEDBACK
Obstacle
• Testing is continuous• State of quality is always visible• Unit testing gives constant feedback to development team
INEFFECTIVE TEST AUTOMATION
Obstacle #4
Ineffective Test AutomationHow do you implement automation… that works?
Example: Financial Institution
• Problem: After automating test scripts, this financial institution kept finding false negatives in bug reports. Figuring out why was slowing down their release cycle – why was this happening?
• Solution: Do more than automate testing – automate the provisioning of your environment in order to stabilize a testing environment that works.
AUTOMATE PROVISIONING OF YOUR ENVIRONMENT
Solution
INEFFECTIVE TEST AUTOMATIONObstacle
• Stable environment means tests won’t return false negatives
DELAYS IN DEVICE READINESS
Obstacle #3
“Delays in Device Readiness”How do you get the latest devices from the manufacturers in your test lab, online and ready for testing before your users upgrade and are impacted?
Example: Hospitality• Problem: Team pre-orders and picks up on launch
day. Limited devices, always a rush to bring new devices online.
• Solution: Vendor managed procurement can get devices before public launch and manage them for readiness.
• Quantified savings of $6,500 per device per year
STOP MANAGING YOUR OWN DEVICES
Solution
DELAYS IN DEVICE READINESSObstacle
• No waiting in line on launch day• Have access to devices before they launch• Cost savings vs. in-house management
TIMELY ACCESS TO MOBILE DEVICES
Obstacle #2
“Timely access to mobile devices”New devices are being released to the market on a daily basis and have shorter lifespan every time new major models are
released.
Example: Telecommunications Provider• Problem: For every device release, staff had to acquire
device and bring devices to each testing group. Lost devices and delays were a major headache. Travel costs and time was prohibitive.
• Solution: Centrally manage devices for testing via the cloud. Test groups can access devices remotely and still ensure the same true end-user experience.
• Eliminated extra costs and scaled their testing abilities.
PUT YOUR DEVICE LAB IN THE CLOUD
Solution
TIMELY ACCESS TO MOBILE DEVICES
Obstacle
• One central device management location• Eliminate procurement delays and costs• Scalable – remote teams can access from anywhere
MANUAL TESTING TAKES TOO LONG
Obstacle #1
“Manual Testing Takes Too Long”Top factor that slows down app lifecycles
Example: Large Bank• Problem: Pre-automated test environment was
heavily reliant on manual testing, and included duplicate labs and a 16-20 day regression cycle.
• Solution: After automating their testing scenarios, they saw a 90% time reduction per regression cycle
• Quantified $40,000 savings per test cycle.
AUTOMATE TESTINGSolution
MANUAL TESTING TAKES TOO LONG
Obstacle
• Create robust scripts to handle testing scenarios• Aim to automate 95% of tests to achieve optimal test coverage• Use an automated testing solution that integrates with your
development environment
6 Ways to Speed Up Testing
1. Implement Automated Testing2. Put your device test lab in the cloud3. Stop managing your own device lab4. Automate provisioning of your
environment5. Implement Continuous Quality6. Evidence is key
www.perfectomobile.comLEARN MORE