mobile app testing is not something special · tools (espresso for android, xctest for ios) mobile...

MobileAppTestingisnotsomethingspecial

Simon‘Peter’Schrijver

TesT-PRO

@simonsaysnomore

[email protected]

simonsaysnomore.wordpress.com

MycareerinMobile(App)Testing

•  Between2006and2014

•  TestManager

•  Didalotoftestexecutionmyself

•  Projects

•  IntroductionofnewMVNO

•  Upgradeofnetworksystems

•  HLR

•  IN

•  CDRprocessing

•  CMS

•  Billing

•  IntroductionofiPhone(2008)

•  DataRoaming


•  BetweenJanuaryandOctober2014

•  SeniorTelecomApplicationTester

•  Projects

•  BuildingHLR

•  AuC

•  HLR

•  IN

•  Automation

•  SOAPUI


•  FromOctober2014upuntilnow

•  SeniorTesterMobileApp

•  Projects

•  Jumbo

•  ABN-AMRO

AMobileDeviceisspecial

•  AsSteveJobssaidduring

theintroductionoftheiPod

“Iwantpeopletocarry

1000songsintheir

pocket”*

•  HelateraddedtheInternet,

TouchandPhonetoit

*Source:http://genius.com/Steve-jobs-ipod-introduction-annotated


•  Youcarryalotofthingswithyouononedevice

•  Itisapartof

everybody'slive

•  Awholenewindustryis

created


•  Becauseitismoving

•  Bringssomenewthingstotest

for

–  Movement

–  Edgeofnetwork

–  Portrait/Landscape

–  Touch

–  Notifications

–  Speedofdata

–  Batteryofdevice

–  Waitingtime


•  Networkinterfaces

–  Bluetooth

–  NearFieldCommunication

–  GlobalPositioningSystem

–  Wifi

–  Cellularnetworks

•  Sensors

FragmentationAndroidin2015

•  Uniquedevices

24093(increaseof28%)

•  Versions

Kitkat(4.4)runson37.8%of

thosedevices

•  1294Androidmanufacturers

•  1.4billionactiveuserper

monthThefirstdevicewentonsaleinOctober,2008

Source:https://androidhub.intel.com/en/posts/peachpellen/Infographic_Android_in_2015.html

Fragmentationbydevice

Source:http://opensignal.com/reports/2015/08/android-fragmentation/

Fragmentationbybrand

Source:http://opensignal.com/reports/2015/08/android-fragmentation/

WhatisaMobileApp?

Amobileapplication(or

mobileapp)isasoftware

applicationdesignedtorunon

smartphones,tablet

computersandothermobile

devices.Source:Wikipedia

TopConsumerMobileApplications

•  MoneyTransfer

•  Location-BasedServices

•  MobileSearch

•  MobileBrowsing

•  MobileHealth

•  MonitoringMobilePayment

•  MobileMusic

•  MobileAdvertising

•  MobileInstantMessaging

Source:http://www.nielsen.com/us/en/insights/news/2015/tops-of-2015-digital.html

TypeofMobileApp?

Thereareprimarilythreetechniquesforcreating

MobileApps:

•  Nativeapplications

•  Mobilewebapplications

•  Hybridmobileapplications

–  Web-to-Appapplications

–  Web-and-Appapplications

TypeofMobileApp?

Native

•  Installedapplicationscanuseall

mobileortabletdevicefeatures

suchascamera,calendar,and

contacts.

•  Supporttherichestuser

experience(forexample,gaming

applications)

•  Builtusingeachphone'snative

SDK

Source:http://www.worklight.com

TypeofMobileApp?

Web

•  Accessibleovertheinternet

withoutinstallingsoftware

•  Usedevicebrowsersto

providenative-looking

applications

•  Builtusingstandardweb

technologiessuchasHTML5,

CSSandJavaScript


TypeofMobileApp?

Hybrid

•  Leveragewebtechnologyand

nativedevicefeatures

•  Installedapplicationsthatusethe

devicebrowsertodisplayweb-

baseduserinterface

•  Abletousenativedevicefeatures

•  Builtusingstandardweb

technologies(HTML5,CSS,JS)

andphone'snativeSDK.


Nativevs.Webvs.Hybrid


WhatisMobileAppTesting?

•  Softwareonamobiledevice

•  Interactwithanuser

•  Senddataoveraninterface

•  Toabackendsystem

•  Waitforaresponse

•  Andprocessthisresponse

WhatisMobileAppTesting?

•  Fragmentation

–  Devices

–  Brand

–  Operatingsystem

•  Featuresonthedevice

•  Native,Hybrid,Web

•  FeaturesoftheApp

MobileAppTesting-Fragmentation

•  Devices

•  Brands


•  Android–  4.0

–  4.1,4.2,4.3

–  4.4

–  5.0,5.1

–  6.0

•  iOS

–  7.0,7.1

–  8.0-8.4

–  9.0–9.3

–  9.4?

–  10.0(sept16)


Howtoassessfragmentation?

•  Definetargetgroupstonarrowyourscope

•  Choosebetweensimulators(emulators)andreal

devices(rentaldevices)

•  Assesstherisksyouarefacing

•  Testsmart


Testapproach(manual)

•  Integration/regressionteston1realAndroidandiOSdevice,

completetestset

•  3-4differentAndroid/iOSdeviceswithreducedtestsetbased

onassessment

•  Timeleftover,testwithotherdevices

Startautomationassoonaspossiblewithnativeautomation

tools(EspressoforAndroid,XCTestforiOS)

MobileAppTesting-Featuresonthedevice

•  Network

•  Camera

•  Battery

•  Touchscreen

•  Voicerecording

•  Location

MobileAppTesting-Native,Hybrid,WebNative

•  Installationtest

•  InterfacewithAPI

•  Touch

•  DifferentOS

•  Interactionwithdevice

attributes,likecamera

MobileAppTesting-Native,Hybrid,WebWeb

•  Reachedbynavigatingtoawebsiteby

wayofanURL

•  Allcodeexecutedfromaserver,

resultinginnetwork-dependent

performance

•  Havetotestdifferentmobilebrowsers.

•  Responsivewebdesign,responds

automaticallytowhicheversizedevice

auserison

MobileAppTesting-Native,Hybrid,WebHybrid

•  BestfromtheNativeworld

andtheWebworld.

•  Accesstodevice-specific

features

MobileAppTesting-Question

Whichoneofthesefourexpressionsdoesnotbelonghere?

•  Usability

•  Mobile

•  Performance

•  Security

Mobileismadeupasaspecialismbythecommercialpeople

Guesswhat?Youcanbecomea“CertifiedMobileTester”(via

ASTQB)

MobileAppTesting-FeaturesoftheApp

Makesuretheuserhasanawesomeexperience

Thatmeans,thefunctionalityoftheappisinorder,

thesecuritymeasuresareinplace,theperformance

oftheappiswithinacceptedlimitsandthe

usabilityoftheappisuserfriendly.

These4qualityattributesmustmeettheexpected

requirements

SkillsforaMobileApptester

Amobiletestermustpossess

certainskillswhowillmakehim/

hervaluable.

Improveyourskillsbylearning

fromotherApps.

Watchhowotherpeopleuse

theirApps

•  Communication

•  Curiosity

•  Criticalthinking

•  Tenacity

•  ConstantLearner

•  Creativity

•  Customerfocus

•  Programmingandtechnical

skills

SkillsforaMobileApptester-Communication

•  Listencarefully.

•  Don’tinterruptotherpeoplewhile

they’respeaking.

•  Speakclearlyandprecisely.

•  Don’tgetpersonalwhen

communicatingwithotherpeople.

•  Beabletocommunicateondifferent

levels,rangingfromtechnicaltonon-

technicalaudiences.

•  Improveyourvocabularybyreading

books,blogs,andnewspapers.

DanielKnott-Hands-OnMobileAppTesting

SkillsforaMobileApptester-Curiosity

Acurioussoftwaretester

exploresthesoftwaretogetas

muchinformationaspossibleout

ofthesystemtoidentify

potentialproblemsandraise

interestingquestionsaboutthe

product

Source:DanielKnott-Hands-OnMobileAppTesting

SkillsforaMobileApptester-CriticalThinking

•  “Criticalthinkingisthinking

aboutthinkingwiththeaim

ofnotgettingfooled.”–

MichaelBolton

•  Gainadeepunderstanding

oftheproduct

•  Focusontherighttesting

work


SkillsforaMobileApptester-Tenacity

Softwaretestersmustbetenacious

andexplainorraisebugsoverand

overagainuntilthesoftwarequality

standardshavebeenmet.Butbe

carefulwiththisasyoumayendup

beingconsideredanuisance.Here

it’simportanttorelyonyourstrong

communicationskills.


SkillsforaMobileApptester-ConstantLearner

•  Softwaretestersneedtotake

noteofchangestakingplace

aroundtheminordertoadapt

andlearnnewapproaches,

methodsandtechnologies.

•  Tokeeppaceandlearnnew

techniquesandtools,software

testerscanreadblogsorbooks

andattendconferencesand

trainingcourses.


SkillsforaMobileApptester-Creativity

•  Thecreativityprocessstarts

withgeneratingthetest

ideaandcollectingthetest

data.

•  Becreativeinordertouse

mobiledevicesindifferent

waysbypayingattentionto

alltheinterfaces,sensors,

andlocations.


SkillsforaMobileApptester-CustomerFocus

•  Trytothinklikeacustomerinorder

todeterminewhetherthesoftware

beingtestedmeetscustomerneeds

•  Requiresyoutobeaproductand

fieldexpertwithinyourteam

•  Talktocustomerstodeterminetheir

needsandproblems

•  Usetheknowledgetoeveryphaseof

thesoftwaredevelopmentprocess


SkillsforaMobileApptester-Programmingskills

•  Mobiletesterswithno

programmingskillsneedtotrain

themselvesby–  readingabookaboutprogramming

languagesorpatterns

–  followingaprogrammingtutorialonthe

Internet

–  attendingaprogrammingcourse.

•  It’salsopossibletoaska

developertotrainthemobile

testerwithinaprojector

company.


MobileAppTestCoverageModel

•  Heuristic:LONGFUNCUP

•  DhanasekarSubramaniam

•  Testcoveragemodelfocussedon

testingofMobileApp

•  Location

•  Orientation

•  Network

•  Gesture

•  Function

•  UserScenarios

•  Notification

•  Communication

•  Updates

•  Platform

https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/

LONGFUNCUP-Location

•  Appshouldbesmartenoughto

trackyourcurrentlocationand

providedsuggestions

surroundingyourlocality

•  Appasksforuserpermission

beforetrackingthelocation

•  Optionstomockyourlocations

fortestingtheapp

•  DonottestanAppsittingatyour

desk


LONGFUNCUP-Orientation

•  Changetheorientation,app

shouldbeabletoprovidea

consistentuserexperience

•  Testallyourscreens,pop-ups,

toastmessages,formsinall

supportedorientations

•  Instancewherethefilledinform

datadisappearswhenyou

changetheorientation


LONGFUNCUP-Network

•  Mobiledevicessupportsboth

cellularandwi-fi.

•  Theycanautomaticallyswitch

betweenanyavailablenetworks.

•  Howdoestheappbehavewhen

thedeviceswitchesbetween

networks?

•  Testforsignalstrength,network

availabilityandreliability


LONGFUNCUP-Gesture

•  Doesyourappsupportsall

standardgestures?

•  Isitconsistentacrosstheapp?

•  Ifitusesanynewgestures,isit

easyfortheuserstounderstand?


LONGFUNCUP-Function

•  Anyfunctionthatdefinesor

distinguishestheproductor

fulfillscorerequirements.

•  Testforinteractions,error

handling,startingandclosingof

theapp,fileaccess,navigation,

multimediaandsync.


LONGFUNCUP–UserScenarios

•  Anyscenarioshouldhaveastory

thatiscredible,motivating,

complexandeasytoevaluate

•  Trytothinkaboutdisfavored

usersandhowtheymighttryto

exploittheapp.

•  Comparewithcompetitororweb

interfacetogetmorereallife

scenarios


LONGFUNCUP-Notifications

•  Notificationsenablean

applicationtoinformits

usersthatithassomething

forthem.

•  Testforallavailabletypes

ofvisualnotifications,

soundandvibration.


LONGFUNCUP-Communication

•  Theprimarypurposeforusing

mobilephonesisvoicecalls.How

doestheappbehaveafter

interruptionsbyanincoming

call?

•  Whatabouttextmessage,Skype

videocallorFaceTime?

•  Testforsuchinterruptionsfrom

voicecommunicationandall

necessaryfunctionsofyourapp.


LONGFUNCUP-Updates

•  Mobileappgetsfrequent

updates.Notonlyappupdates

butOSupdatesaswell.

•  Whatchanges,whichnew

featuresareinthelatestOS

update

•  Analyseiftheappneedsany

modificationbecauseoftheOS

updates.


LONGFUNCUP-Platform

•  Haveagoodunderstandingof

thepopularmobileplatforms

especiallyAppleandAndroid

•  Beawareofthehistoryandalso

thelatesttrendsinthemobile

platform.

•  Whattoolsareavailablefor

testing

•  Whatkindoftest-abilitylayers

areprovidedbytheplatform


Summertimebug

ViathebrowserIenteredmybirthday,onthemobile

appIdisplayedmybirthday.Itwasdisplayedwrong

https://simonsaysnomore.wordpress.com/2016/03/08/the-summertime-bug/

Nextyear

Usefulinformation

•  http://katrinatester.blogspot.co.nz/2015/08/mobile-testing-pathway.html

•  http://adventuresinqa.com/2016/05/03/10-mobile-app-testing-mistakes-to-avoid/

•  http://adventuresinqa.com/2015/06/22/top-10-best-practices-for-mobile-app-

testing/

MobileApptestingisnotsomethingspecial

•  Mobiledeviceisachannel.Theusergainsaccessvia

anApptothebackendwherealltheinformationis

stored.

•  MobileApptestingisbasedonfourquality

attributes.Functionalityisthemostimportantone.

•  Whenyouasatesteralreadyhavesometest

experience,youcandoMobileApptesting

I’maTester

•  50yearsold

•  Testersince1997

•  TestMobileAppssinceNovember2014

•  GottheMobileAppstestingjobviamynetwork

•  Basedonmyreputationasatester

•  Myjobinterview(therewasnone)

•  LearntotestMobileAppsjustbymyself

SimonSaysThankYouDoyouhaveanyquestions?

@simonsaysnomore

[email protected]

simonsaysnomore.wordpress.com

mobile app testing is not something special · tools (espresso for android, xctest for ios) mobile...

Documents