agile mobile testing workshop
DESCRIPTION
Find out what testing works for your mobile app. Agile Software Development means we want to maximise progress while minimising waste. Delays cause waste, for instance wasted time and efforts; ineffective work causes waste; poor quality causes waste; and bugs cause waste and delay progress, etc. Mobile apps and the mobile app ecosystem help determine what sorts of testing will be more valuable for the project. This workshop introduces various key concepts and factors related to testing mobile apps effectively. You will have the opportunity to practice testing mobile apps during the workshop to help reinforce your learning and discovery. We will cover both interactive and automated testing of mobile apps, and find ways to reduce the Time To Useful Feedback (TTUF) so the project team can make more progress while reducing project waste. We will also cover various ways to gather more and better information about the qualities of our mobile codebase and of the quality of the apps-in-use. Bring your mobile apps and mobile devices and be prepared to get involved in testing! More details: http://confengine.com/agile-pune-2014/proposal/861/agile-mobile-testing Conference: http://pune.agileindia.org/TRANSCRIPT
![Page 1: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/1.jpg)
AGILE MOBILE TESTING WORKSHOP PUNE AGILE CONFERENCE
JULIAN HARTY Contact me: [email protected] Rev: 22 Nov 2014
Creative Commons License How to design your mobile apps by Julian Harty is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License. http://creativecommons.org/licenses/by-sa/3.0/deed.en_US
![Page 2: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/2.jpg)
AGILE TESTING
![Page 3: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/3.jpg)
http://lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/
![Page 4: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/4.jpg)
TTUF
TIME TO USEFUL FEEDBACK
Information is more valuable when it is timely
![Page 5: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/5.jpg)
CONTINUOUS INTEGRATION (FOR MOBILE APPS) Raw Ingredients • Code • Code Repository (git, svn, …) • Triggers • Build tools • Automated tests • Run time environment(s)
• Emulators • Simulators • Devices
![Page 6: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/6.jpg)
WHERE ARE THE LATENCIES
WORKSHOP
![Page 7: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/7.jpg)
WHERE ARE THE LATENCIES? • Build times
• Affects end-to-end Unit Test runtime • Commissioning run-time environment
• Automated tests • Deployment
• Installing the app so it can be tested • App Store Approval
• Feedback from the market • Feedback from the field
• App Qualities • Failures & Defects in use
![Page 8: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/8.jpg)
INTERACTIVE TESTING
TTUF
![Page 9: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/9.jpg)
TBS
We want to maximize T and minimize B & S
T B S
T = Testing B = Bug reporting S = Setup
![Page 10: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/10.jpg)
MINIMIZE SETUP Email or SMS URLs to phone
Have a configuration workstation with all the drivers installed
Create apps on your build server and make them available
![Page 11: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/11.jpg)
MINIMIZE BUG INVESTIGATION
• Screenshot utilities
• Learn how to access, filter and store device logs
• Good quality camera for close-up screenshots
• Write a bug report that will still be valuable when the bug will actually be investigated
![Page 12: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/12.jpg)
MAXIMIZE TESTING Use testing heuristics
• I SLICED UP FUN (Jonathan Kohl)
• COP FLUNG GUN (Moolya)
![Page 13: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/13.jpg)
HEURISTICS & MNEMONICS
ANTIFRAGILE TESTING
![Page 14: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/14.jpg)
http://moolya.com/blogs/2012/04/121/Test-Mobile-applications-with-COP-who-FLUNG-GUN
![Page 15: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/15.jpg)
TEST THIS
Kiwix
USING THIS GUIDE
http://moolya.com/blogs/
WiFi Password: FW4WFAAA
![Page 16: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/16.jpg)
AUTOMATED TESTS
TTUF
![Page 17: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/17.jpg)
REDUCE EFFORT
TESTABILITY
![Page 18: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/18.jpg)
TESTABILITY
SPENDING MONEY WISELY
![Page 19: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/19.jpg)
WHAT IS TESTABILITY? The concept of designing & implementing software so it is easier to test • Testing can be automated • Testing can be interactive
![Page 20: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/20.jpg)
SCALES OF TESTABILITY
There are at least 2 dimensions of Testability: • ease of interfacing • transparency into the state & behaviour of the software being tested.
easy
transparency
inte
rfaci
ng
challenging
trans
pare
nt
opaq
ue
![Page 21: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/21.jpg)
DESIGNING FOR TESTABILITY: HOOKS Programmatic Hooks To connect test automation easily Consider whether to leave them in situ
![Page 22: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/22.jpg)
DESIGNING FOR TESTABILITY: VISIBILITY
“Eyes into the Soul of the machine...” Expose internal data and state
• Makes some checks easier to confirm • e.g. Error recovery mechanisms cleaned up the
app’s internal state Beware:
• Non-test code might start using the data • If so, consider formalising the access in an API
![Page 23: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/23.jpg)
TESTABILITY: LAYERING OF CODE Already covered some aspects in the Segmented Design topic Ideal to be able to automate the testing of each layer or component independently Then testing of the composite software can focus on testing the composite aspects Beware of emergent behaviour
• Test the qualities: non-functional-testing (NFT)
![Page 24: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/24.jpg)
TESTABILITY: SEPARATION OF CONCERNS
Separate generic and platform-specific code Generic code:
• Application logic: What the app does, functionality Platform-specific code:
• User Interface • Threading • Calls to platform-specific APIs
![Page 25: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/25.jpg)
TESTABILITY: ISOLATE COMPLEX CODE
Try encapsulating & isolating complex code • Provide an interface* • Have excellent automated tests exercise it • Warn casual developers (and testers) not to tamper
with it • Now the rest of our code is easier to understand &
manage In parallel consider ways to replace complex code with simpler code
* e.g. See the Facade design pattern
![Page 26: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/26.jpg)
BACK TO “VALUE”
BIG PICTURE
![Page 27: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/27.jpg)
FULL LIFECYCLE COSTS
SPENDING WISELY?
![Page 28: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/28.jpg)
FULL LIFECYCLE COSTS The initial development effort may be dwarfed by maintenance work There are trade-offs between reducing the cost of initial development and the cost of maintenance work Code that costs more to modify is undesirable. Well designed code & good automated tests can reduce the risk and cost of maintenance work.
Beware of premature aging of your app’s codebase!
![Page 29: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/29.jpg)
SPEND MONEY ON TESTING?
WHERE AND WHEN TO
![Page 30: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/30.jpg)
NOVODA Costs 60% more to ‘add’ test automation to Android projects
Who’s willing to sign off on it?
Where and when does the ROI start?
![Page 31: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/31.jpg)
THINGS TO CONSIDER How long do your code bases ‘last’? Who pays for ‘maintenance’? Where is the expertise to maintain the code? Active apps need ongoing nurture & investments even if you’re not changing the functionality
![Page 32: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/32.jpg)
ALTERNATIVES TO TESTING Testing is not the only way to obtain useful feedback. Sometimes it’s not the best way either.
![Page 33: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/33.jpg)
COMPLEMENTING TESTING WITH OTHER INFORMATION SOURCES
• Crowd Sourcing • Log Analysis & Crash Dumps • Analytics • In-app feedback
![Page 34: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/34.jpg)
VISUALIZATION TOOLS
UiAutomationViewer (for Android) Using visualisation tools to help define the test automation interface
![Page 35: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/35.jpg)
USING MOBILE ANALYTICS
SECTION 7
![Page 36: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/36.jpg)
USING MOBILE ANALYTICS An overview of Mobile Analytics How they can help augment our testing
![Page 37: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/37.jpg)
TOPOLOGY
Overview of Mobile Analytics Each step may be delayed
Dat
a C
olle
ctor
Database
Analytics WebServer
Business view
Filter(s)
Mobile Apps sending Analytics data
![Page 38: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/38.jpg)
TYPES OF EVENTS
Mobile app Analytics Library
Analytics Collector
1:1 App-initiated event
m:1 App-initiated event
Library-initiated event
E1
E
E4 …
L
E
Ea
L
E
Ea AnalyticsDatabase
Internet connection
![Page 39: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/39.jpg)
ANALYTICAL QUESTIONS
Engineering Activity, Benchmarking,
Testing
Trends, Defect Reports Extrapolation
Software quality models, bottleneck
analysis
Specification refinement, asset
reallocation
Failure prediction models
What’s Happened? (Reporting)
What’s Happened? (Alerts)
What will Happen? (Forecasting)
How and why did it happen?
(Factor analysis)
What is the next best action?
(Recommendation)
What’s the best / worst that can happen?
(Modeling / Simulation)
Information
Insight
Past Present Future
![Page 40: Agile Mobile Testing Workshop](https://reader033.vdocuments.us/reader033/viewer/2022052622/559442ee1a28ab01308b4694/html5/thumbnails/40.jpg)
FISHBONES
Feasible Practical Useful