mobile testing - meetupfiles.meetup.com/15864632/mobile testing.pdf · non+issues+ uites6ng+...

Mobile Testing Open Source Solu,ons

Top Q – Who are we? General•  Established in 2005, the leading test-‐automa6on solu6ons company in Israel•  More than 100 customers in major ver6cal markets, including Networking

equipment manufacturers and Java-‐based vendors Methodology•  Deploys a tried-‐and-‐true project development methodology, proven to build

successful and enduring test-‐automa6on projects Tools•  Exper6se in all leading test-‐automa6on plaHorms and tools, including commercial

to open source Services•  Complete life-‐cycle of a test-‐automa6on solu6on: from ini6al requirement phase,

analysis and planning, to implementa6on, maintenance and audit

Customers

Non Issues UI Tes6ng

Fragmenta6on Tools Quick Review instrumenta6on

Calabash UiAutomator

Appium Case Studies We’re not going to talk about

Tes6ng framework Programming language

BDD, TDD, ATDD,KDT etc.




Appium Case Studies

+

Image Based Approach




Appium Case Studies UI Tes,ng

UI

Integra6on

Unit




Appium Case Studies UI Tes,ng

•  Func6onal black box tes6ng – no need of internal implementa6on details of the app, only its expected output when a user performs a specific ac6on or enters a specific input.

•  test the behavior of your applica6on’s user interface (UI) when it is running on a device

•  Usually 6me-‐consuming, tedious, and error-‐prone when done manually




Appium Case Studies Fragmenta,on Issues




Appium Case Studies UI Tes,ng – Commercial

ConsProsPlaIormAPIFrameworkWorks with all devices Works with IOS on PC

Android/IOSJava / C# / Perl/Python

SeeTest

Selenium API Works with IOS on PC A lot of tes6ng devices

Android/IOSRecording Selenium API

Perfecto Mobile

DDT supports web and

desktop app

Android/ios/ windows mobile

.netRanorex

Good KDT Excellent repor6ng

Supports DB,Web,Desktop apps

etc.

AndroidC#/Java Script

TestComplete

Works with other Mobile Automa6on

Framworks

AndroidVB.netQTP




Appium Case Studies UI Tes,ng – Open Source

ConsProsPlaIormAPIFrameworkSinging

Instrumenta6ons App domain only

Works with all Android versions

Hybrid Apps

AndroidJava/C#/ Python etc.

Robo6um

Android API 16 and above

Selenium API Works with IOS Hybrid Apps

Android/IOSJava/C#/ python etc. Selenium API

Appium

Android API 16 and above

No WebView control

No need of special permission

Controls the whole UI Hybrid Apps

AndroidJava/C#/ python etc.

UI Uiautomator

not enough documenta6on

Easy to useIOSJava/ cucumber

Frank

BDDBDDIOS/AndroidCucumber+ruby/java

Calabash (Xamarin)

No mobile webGoogleAndroidJavaEspresso




Appium Case Studies UI Tes,ng – Commercial

ConsProsPlaIormAPIFrameworkSelenium API

Works with IOS on PC A lot of tes6ng devices

Android/IOSRecording Selenium API

Perfecto Mobile

Based on Appium Runs in the cloud

Android/IOSJava/Ruby/ PHP/JS/ Python

SauceLabs

Excellent UI tes6ng toolAndroidJava/Ruby/ PHP/JS/ Python

Applitools

Real devices on cloudAndroid/IOSJava/ Cucamber

TestDroid

Based on CalabashAndroid/IOSC#Xamarin Test Cloud




Appium Case Studies

•  A service provided by the different opera6ng systems

•  Designed for unit tes6ng •  Interfaces with the applica6on UI objects

Instrumenta,on Based Approach




Appium Case Studies Instrumenta,on Based Approach

•  The tes6ng applica6on must have instrumenta6on permissions on the applica6on under test package

•  In Android we need to use the same signature in the DUT and the test applica6on

•  Provided differently in each OS •  Enable to test only the applica6on context




Appium Case Studies Instrumenta,on Based Approach




Appium Case Studies

•  Open source library extending JUnit •  Black-‐box test cases for Android apps •  Supports na6ve, hybrid and mobile web tes6ng

•  Semi-‐ac6ve open source community with monthly releases

Robo,um




Appium Case Studies Robo,um – example

public class NotePadTest extends ActivityInstrumentationTestCase2<NotesList>{ private Solo solo;

@Override public void setUp() throws Exception {

//setUp() is run before a test case is started. //This is where the solo object is created.

solo = new Solo(getInstrumentation(), getActivity()); {

public void testAddNote() throws Exception { solo.clickOnMenuItem("Add note"); //Assert that NoteEditor activity is opened solo.assertCurrentActivity("Expected NoteEditor activity", "NoteEditor"); solo.enterText(0, "Note 1"); solo.goBack(); solo.clickOnMenuItem("Add note"); solo.enterText(0, "Note 2"); solo.goBackToActivity("NotesList"); //Takes a screenshot and saves it in "/sdcard/Robotium-‐Screenshots/". solo.takeScreenshot(); boolean expected = true; boolean actual = solo.searchText("Note 1") && solo.searchText("Note 2");

assertEquals("Note 1 and/or Note 2 are not found", expected, actual); }

}




Appium Case Studies

•  Robo6um remote control allows execu6on of Robo6um tests from the deskop

•  No need to deploy tests with the applica6on •  Java and Python binders

Robo,um Remote Control

Robo6um RC

Robo6um Client

Instrumenta6on

Json over HTTP

Test Code




Appium Case Studies Calabash

•  Behavior driven test framework for na6ve Android, na6ve iOS and mobile web (Android)

•  Calabash installs an HTTP server as an instrumenta6on package that listens commands from Calabash server

•  New controllers can be implemented in Ruby or Java •  Tests are executed on server side •  Each test scenario is described in Cucumber •  Ruby Client library converts Cucumber commands to

either Robo6um or Frank method calls




Appium Case Studies Calabash

iOS Android




Appium Case Studies Calabash -‐ Gherkin




Appium Case Studies UiAutomator

•  A Java library, created by Google, containing APIs to create automated customized func6onal UI tests.

•  An execu6on engine to run the tests. •  UiAutomator allows to write code based UI Tests with no screen resolu6on dependency

•  UiAutomator will work on any Android device from API 16 and above




Appium Case Studies UiAutomatorViewer

•  A GUI tool to scan and analyze the UI components of an Android applica6on.

•  Inspect the UI of an applica6on in order to find the layout hierarchy and view the proper6es of the individual UI components of an applica6on

•  Aher the installa6on of Android SDK, the tool exists in the /tools/ folder and you can start it by typing: uiautomatorviewer from the command line.




Appium Case Studies UiAutomatorViewer -‐ Example

•  A screenshot of the Calculator” applica6on opens. •  For example click on “7” bulon and look at the right bolom panel the

“resource-‐id” value




Appium Case Studies Wri,ng a Test – code example

public class CalculatorTest extends UiAutomatorTestCase {

public void tes6ngCalculator() throws UiObjectNotFoundExcep6on { getUiDevice().pressHome(); UiObject Applica6ons = new UiObject(new UiSelector().descrip6on("Apps")); Applica6ons.clickAndWaitForNewWindow(); UiObject apps = new UiObject(new UiSelector().text("Calc")); apps.click(); UiScrollable ListOfapplica6ons = new UiScrollable(new UiSelector().scrollable(true)); UiObject Calculator = ListOfapplica6ons.getChildByText(new UiSelector().className(android.widget.TextView.class.getName()), "Calculator"); Calculator.clickAndWaitForNewWindow(); UiObject seven = new UiObject(new UiSelector().resourceId("com.android.calculator2:id/digit7")); seven.click(); UiObject plus = new UiObject(new UiSelector().resourceId("com.android.calculator2:id/plus")); plus.click(); UiObject one = new UiObject(new UiSelector().resourceId("com.android.calculator2:id/digit1")); one.click(); UiObject result = new UiObject(new UiSelector().resourceId("com.android.calculator2:id/equal")); result.click(); getUiDevice().pressBack();

{{




Appium Case Studies Running a Test

•  Type <sdk_path>/tools/android create <name> -‐n -‐t 5 -‐<path> The <name> is the name of the project that contains your uiautomator test source files, the <path> is the path to the corresponding project directory.

•  In order to find the correct value for the parameter -‐t, type android list targets.

•  Go to your project directory and type ant build •  Push the JAR file to your Android device •  Run the tests by typing

adb shell uiautomator runtest <Path/Jar_name> -‐c <package.test_class> •  for example:

adb shell uiautomator runtest LaunchSe>ngs.jar -‐c com.uia.tests.LaunchSe>ngs




Appium Case Studies UiAutomator Server – Client Solu,on

•  As can see from the pervious slides the UiAutomator code is compiled and pushed and executed on the DUT

•  To allow more flexible test code (I.e test paramteriza6on) and to enjoy the test framework service Top Q’s solu6on is based on client-‐server architecture over HTTP implanted with JSON-‐RPC

•  The server contains two compiled jars and it is running on the DUT with the regular UiAutomator execu6on command

•  The server implements an interface that refelects UiAutomator API •  The client is a lean Java class that uses the same interface over JSON-‐RPC

client •  UiSelector, UiDevice and UiObject are implemented in both sides




Appium Case Studies Appium

•  Allows to locate UI elements and perform various opera6ons of them

•  Supports Mac and Windows •  Supports na6ve, hybrid and web applica6ons •  Supports iOS and Android (and FirfoxOS) •  Supports real devices and simulators •  Uses Selenium Webdriver (W3C standard) as a scrip6ng framework




Appium Case Studies Appium Architecture

•  Appium is an HTTP server that creates and handles WebDriver sessions

•  It starts an Appium server on the device that is listening commands from the main Appium process

•  It receives json requests from client libraries over HTTP

•  On Android Appium executes these commands as either uiautomator or Selendroid commands depending on the API level of the device

•  Tests are driven from a Selenium script on Appium server




Appium Case Studies Android Architecture




Appium Case Studies iOS Architecture




Appium Case Studies Appium inspector




Appium Case Studies

DesiredCapabilities capabilities = new DesiredCapabilities(); capabilities.setCapability("platformName", "Android"); capabilities.setCapability("app","src/test/resources/com.twitter.android_5.5.0.apk"); capabilities.setCapability("app-‐package", "com.twitter.android"); capabilities.setCapability("app-‐activity", ".LoginActivity"); AppiumDriver driver = new AppiumDriver(new URL("http://127.0.0.1:4723/wd/hub"), capabilities);

Appium API

//Find the element in page using a selector locating strategy WebElement element = driver.findElement(MobileBy.className("android.widget.EditText")); //Perform operation on the found element element.sendKeys("jsystemdemo");




Appium Case Studies Addi,onal Test Tools

•  Fiddler Core – Based on fiddler core DLLs –  Java API

•  Logcat parsing -‐  Wait for specific message -‐  Search for excep6ons

•  GennyMo6on – Android Emulator •  SDK tes6ng

– Tester appliac6on – Client – Server architecture with JSON-‐RPC




Appium Case Studies Page Object Design Pa\ern

•  Within your applica6on UI there are areas that your tests

interact with.

A Page Object simply models these as objects within the

test code.

•  The Page Object’s methods represent the services offered

by a par6cular page.

•  The page object encapsulate the UI elements. if the UI

changes, the fix need only be applied in one place.




Appium Case Studies UI Repository

•  In code or in test framework GUI use repository keys instead of real locators

•  Saves 6me between versions •  In some cases can be used with iOS/Android versions




Appium Case Studies UI Repository -‐ GUI




Appium Case Studies UI Repository -‐ Code

public class LoginTests { private String username; private String password; @Test @TestProperties(name="Login successfully with username ${username} and password ${password}", paramsInclude = { "username","password" }) public void loginSuccess() throws Exception{ appiumSo.getActions().waitForElement(appiumSo.objectRepository.getObject("login.username"),30); appiumSo.getActions().sendText(appiumSo.objectRepository.getObject("login.username"), username); appiumSo.getActions().sendText(appiumSo.objectRepository.getObject("login.password"), password); appiumSo.getActions().click(appiumSo.objectRepository.getObject("login.btn")); appiumSo.getActions().waitForElement(appiumSo.objectRepository.getObject("fileManager.switchBtn"),30); }

Thank you For listening

Nir Armon Top Q [email protected] Find me on LinkedIn

mobile testing - meetupfiles.meetup.com/15864632/mobile testing.pdf · non+issues+ uites6ng+...

Documents