6 ways to speed up app testing

27
6 Ways to Speed Up App Testing

Upload: perfecto-mobile

Post on 06-Aug-2015

721 views

Category:

Software


0 download

TRANSCRIPT

Page 1: 6 Ways to Speed Up App Testing

6 Ways to Speed Up App Testing

Page 2: 6 Ways to Speed Up App Testing

Presenter

Christopher P. WillisCMO – Perfecto Mobile

Over 18 years of experience growing companies in the technology sector.

A mobile industry pioneer, Chris helped set the direction for the marketing of mobile applications and mobile application development platforms over a decade ago.

Page 3: 6 Ways to Speed Up App Testing

Setting the bar for market success

Introducing Perfecto Mobile

Perfecto Mobile at a glance

Industry standout & rising star

FROST AND SULLIVAN CUSTOMER VALUE LEADERSHIP AWARD –MOBILE APPLICATION TESTING, GLOBAL, 2013

13Data Centers

50KTesting Hours / Month

99.5%Uptime

Global Coverage

Perfecto Mobile helps dev and test teams deliver continuous quality, accelerating the velocity of app delivery without compromises.

Page 4: 6 Ways to Speed Up App Testing

Demographics

350 Mobile Developers, Testers and QA Managers

Testing / QA71 %

Development 20 %Head of Mobile 9 %

Roles

1 – 5 apps48 %

6 – 10 apps11 %

11 – 20 apps16 %

20+ apps25 %

Development Scope

Page 5: 6 Ways to Speed Up App Testing

Current Process

Agile ProcessesWaterfall

86 %14 %

94 %

Now

Goal

Page 6: 6 Ways to Speed Up App Testing

WHAT TESTING FACTORS SLOW DOWN YOUR RELEASE CYCLES?

Question:

Page 7: 6 Ways to Speed Up App Testing

What testing factors slow down your release cycles?

?Rank what causes

delays during mobile app testing

& release

1. Manual testing takes too long.

2. Timely access to mobile devices.

3. Delays in device lab readiness.

4. Ineffective test automation.

5. Slow QA to Development feedback.

6. Too many defects declared non-

reproducible.

Page 8: 6 Ways to Speed Up App Testing

TOO MANY DEFECTS ARE DECLARED NON-REPRODUCIBLE

Obstacle #6

Page 9: 6 Ways to Speed Up App Testing

Too many defects are non-reproducible

If it’s hard to reproduce defects due to lack of evidence, it is hard to provide sufficient feedback to Dev teams.

Example: An international telecoms customer• Problem: Dev teams had difficultly replicating bugs

that offsite testing teams reported

• Solution 1: Video reporting that recorded entire test transactions was made sharable between Dev and Teams to close this loop

• Solution 2: Replicate the defect on the same device on which it was uncovered

Page 10: 6 Ways to Speed Up App Testing

EVIDENCE IS KEYSolution

TOO MANY DEFECTS ARE DECLARED NON-REPRODUCIBLE

Obstacle

• Video recordings shed light on bugs• Sharable video bridges Dev and Test teams during feedback

cycles, and dramatically improves velocity• Real devices, in the cloud, allow testers and developers to

collaborate over the same device, under the same user conditions

Page 11: 6 Ways to Speed Up App Testing

SLOW QA TO DEV. FEEDBACKObstacle #5

Page 12: 6 Ways to Speed Up App Testing

Slow QA to Developer FeedbackFifth factor, but slows down the cycle more than other

factors.

Example: Travel Site• Problem: Monthly release cycle, with 3 weeks of

development and 1 week for testing and bug fixing. As much work needed to happen in this 1 week as the previous 3.

• Solution: Embedded quality and testing throughout the development cycle. Unit tests make QA feedback visible and developers know where there code stands at all times.

Page 13: 6 Ways to Speed Up App Testing

CONTINUOUS QUALITYSolution

SLOW QA TO DEVELOPER FEEDBACK

Obstacle

• Testing is continuous• State of quality is always visible• Unit testing gives constant feedback to development team

Page 14: 6 Ways to Speed Up App Testing

INEFFECTIVE TEST AUTOMATION

Obstacle #4

Page 15: 6 Ways to Speed Up App Testing

Ineffective Test AutomationHow do you implement automation… that works?

Example: Financial Institution

• Problem: After automating test scripts, this financial institution kept finding false negatives in bug reports. Figuring out why was slowing down their release cycle – why was this happening?

• Solution: Do more than automate testing – automate the provisioning of your environment in order to stabilize a testing environment that works.

Page 16: 6 Ways to Speed Up App Testing

AUTOMATE PROVISIONING OF YOUR ENVIRONMENT

Solution

INEFFECTIVE TEST AUTOMATIONObstacle

• Stable environment means tests won’t return false negatives

Page 17: 6 Ways to Speed Up App Testing

DELAYS IN DEVICE READINESS

Obstacle #3

Page 18: 6 Ways to Speed Up App Testing

“Delays in Device Readiness”How do you get the latest devices from the manufacturers in your test lab, online and ready for testing before your users upgrade and are impacted?

Example: Hospitality• Problem: Team pre-orders and picks up on launch

day. Limited devices, always a rush to bring new devices online.

• Solution: Vendor managed procurement can get devices before public launch and manage them for readiness.

• Quantified savings of $6,500 per device per year

Page 19: 6 Ways to Speed Up App Testing

STOP MANAGING YOUR OWN DEVICES

Solution

DELAYS IN DEVICE READINESSObstacle

• No waiting in line on launch day• Have access to devices before they launch• Cost savings vs. in-house management

Page 20: 6 Ways to Speed Up App Testing

TIMELY ACCESS TO MOBILE DEVICES

Obstacle #2

Page 21: 6 Ways to Speed Up App Testing

“Timely access to mobile devices”New devices are being released to the market on a daily basis and have shorter lifespan every time new major models are

released.

Example: Telecommunications Provider• Problem: For every device release, staff had to acquire

device and bring devices to each testing group. Lost devices and delays were a major headache. Travel costs and time was prohibitive.

• Solution: Centrally manage devices for testing via the cloud. Test groups can access devices remotely and still ensure the same true end-user experience.

• Eliminated extra costs and scaled their testing abilities.

Page 22: 6 Ways to Speed Up App Testing

PUT YOUR DEVICE LAB IN THE CLOUD

Solution

TIMELY ACCESS TO MOBILE DEVICES

Obstacle

• One central device management location• Eliminate procurement delays and costs• Scalable – remote teams can access from anywhere

Page 23: 6 Ways to Speed Up App Testing

MANUAL TESTING TAKES TOO LONG

Obstacle #1

Page 24: 6 Ways to Speed Up App Testing

“Manual Testing Takes Too Long”Top factor that slows down app lifecycles

Example: Large Bank• Problem: Pre-automated test environment was

heavily reliant on manual testing, and included duplicate labs and a 16-20 day regression cycle.

• Solution: After automating their testing scenarios, they saw a 90% time reduction per regression cycle

• Quantified $40,000 savings per test cycle.

Page 25: 6 Ways to Speed Up App Testing

AUTOMATE TESTINGSolution

MANUAL TESTING TAKES TOO LONG

Obstacle

• Create robust scripts to handle testing scenarios• Aim to automate 95% of tests to achieve optimal test coverage• Use an automated testing solution that integrates with your

development environment

Page 26: 6 Ways to Speed Up App Testing

6 Ways to Speed Up Testing

1. Implement Automated Testing2. Put your device test lab in the cloud3. Stop managing your own device lab4. Automate provisioning of your

environment5. Implement Continuous Quality6. Evidence is key

Page 27: 6 Ways to Speed Up App Testing

www.perfectomobile.comLEARN MORE