testdroid: release perfect apps with mobile visual testing in the cloud

Public Device Cloud

on-demand devices

(multitenant)

Hosted by Bitbar

Mobile app testing on literally any real Android and iOS devices

Private / Reserved Device Cloud

Hosted by Bitbar in the US and/or

Europe

Devices chosen by and reserved only for the Customer

On-Premise Device Cloud

Automated mobile app testing devices

hosted by the customer, usually 30-500 devices

TESTDROID – 3 DEPLOYMENT OPTIONS

Testdroid Cloud Testdroid Enterprise Testdroid PrivateCloud

The Value of ‘Cloud Testing’• Devices / Hardware

- No need to handle shipping, logistics, handling of new devices and other new HW used in the infrastructure monthly

• Infrastructure- Ready-to-Go infrastructure means no need for investments,

significantly speeds things up & saves time and money

• Support- SW devs, SW devs in test (SDET), QA engineers – all need different

type of support when using the infrastructure

• Operations- Updating infrastructure, devices, networks, additional SW

• Tools Integration - Support for multiple test tools gives the users/developers

freedom of choice. No vendor, tech, tools lock-ins!

Testing ‘in the Cloud’

1

2

3

Android Fragmentation 2015

by Open Signal report August 2015

Testdroid Cloud

>95%

Android & iOS OS Flakiness

iOS 8.0iOS 8.0.2iOS 8.1iOS 8.1.1iOS 8.2iOS 8 ALLKitKat 4.4KitKat 4.4.2KitKat 4.4.3KitKat 4.4.4KitKat ALLLollipop 5.0Lollipop 5.0.1Lollipop 5.0.2Lollipop ALLAndroid M

0 5 10 15 20 25 30 35Failed test runs. Percentage (%).

Why Apps Fail?

Why Real Devices Are Must-to-Have

• Emulators cannot help you to test...

• User Experience • Usability• Hardware• Software• Infrastructure

0 % = the percentage of your app users that use emulator to run your app!

Top 5 Pain PointsFragmentationDiversity of mobile platforms creates unique challenges for mobile

app, game and web devs.

Low Quality Apps Lose MoneyUsers uninstall and abandon apps/services if the apps do not work.

Manual Testing on Real DevicesExpensive, slow and error-prone process

Device AcquisitionAcquiring devices for devand testing can be expensive and

practically impossible

InefficienciesDistributed Teams, duplications of HW resources, app diversity,

fragmented tool support.

Manual vs. AutomationManual Automation

Manual vs. Automation

Smaller coverage, More money burnt & time wasted, Error-prone

Manual AutomationLarge coverage,

quickly completed,

Less money & time wasted, Exact results.

Test Automation FrameworksRobotium uiautomato

rEspresso Appium Calabash

Android Yes Yes Yes Yes Yes

iOS No No No Yes Yes

Mobile web Yes (Android)

Limited to x.y clicks

No Yes (Android & iOS)

Yes (Android)

Scripting Language

Java Java Java Almost any Ruby

Test creation tools

Testdroid Recorder

UI Automator viewer

Hierarchy Viewer

Appium.app CLI

Supported API levels

All 16 => 8, 10, 15- All All

Community Contributors Google Google Active Pretty quiet

APPIUM / SELENIUM IN THE CLOUD WITH REAL DEVICES & REAL BROWSERS

Client Side Appium at Testdroid Cloud

Test Script

Test Case

Desired Capabilities

{ “device”: “Android”, “app”: “/Users/user/ApiDemos.apk” “app-package”: “com.example.android.apis” “app-activity”: “.ApiDemos”}

Test Script

Test Case


WebDriverhttp://localhost:4723/wd/hub

Appium Server

4723

Device

Localhost(DesiredCaps)

http://localhost:4723/wd/hub

*Testdroid Caps

http://appium.testdroid.com/wd/hub

(DesiredCaps)

Test Script

Test Case


WebDriver

From Localhost to Testdroid

Client Side ExecutionGo to

cloud.testdroid.com

Client Side ExecutionGet a

Device NameGo to

cloud.testdroid.com

Client Side ExecutionAdd Testdroid Desired

Caps to test script

{ “testdroid_username”: “[email protected]”, “testdroid_password”: “p4s$w0rd”, “testdroid_project”: “My First Project”, “testdroid_testrun”: “Test 1”, “testdroid_device”: “iPad Mini 7.0.4 A1432”, “testdroid_app”: “http://domain.com/app_v1.ipa” . . “app”: “com.bitbar.testdroid.BitbarIOSSample”}

Get a Device Name

Go tocloud.testdroid.com

Client Side Execution

driver = webdriver.Remote("http://appium.testdroid.com/wd/hub", desired_caps);

Point the Webdriver to http://appium.testdroid.com/wd/hub

Add Testdroid Desired Caps to test script

Get a Device Name



Run the Test ScriptGet Results from Testdroid Cloud



Get a Device Name


ScreenshotsLogs

Detailed results


Pull the Results from the Result URL

driver = webdriver.Remote("http://appium.testdroid.com/wd/hub", desired_caps);

Run the Test ScriptGet Results from Testdroid Cloud



Get a Device Name


Multiple Devices – Client Side

python testscript.py –device <devicename>1

python testscript.py –device <devicename>2

python testscript.py –device <devicename>n

Test Script

Test Cases

Instigator Script

deviceArray=[ “iPad 4 6.0.1 A1458”, “iPad mini 7.0.4 A1432”, . . . “iPhone 4S 6.1.3 A1387”,]

TESTDROID CLOUD AT GLANCE

THANK YOU!More information about frameworks,

devices and testing at www.testdroid.com