mobile application testing af frederik carlier, jamo solutions
DESCRIPTION
Oplægget blev holdt ved et seminar i InfinIT-interessegruppen Softwaretest den 11. september 2013. Læs mere om interessegruppen her: http://infinit.dk/dk/interessegrupper/softwaretest/softwaretest.htmTRANSCRIPT
![Page 1: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/1.jpg)
Mobile Application TestingFrederik Carlierww Technology Strategist
11th September 2013
![Page 2: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/2.jpg)
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
![Page 3: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/3.jpg)
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
![Page 4: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/4.jpg)
2013 Gartner CIO PrioritiesRanking 2013 2012 2011 2010 2009 2008
Analytics and Business Intelligence 1 1 5 5 1 1
Mobile Technologies 2 2 3 6 12 12
Cloud Computing 3 3 1 2 16 *
Collaboration Technologies (workflow) 4 4 8 11 5 8
Legacy Modernization 5 6 7 15 4 4
IT Management 6 7 4 10 * *
Customer Relastionship Management 7 8 18 * * *
Virtualization 8 5 2 1 3 3
Security 9 10 12 9 8 8
ERP Applications 10 9 13 14 2 2
* Not an option this yearSource: http://www.gartner.com/imagesrv/cio/pdf/cio_agenda_insights2013.pdf
![Page 5: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/5.jpg)
2013 Gartner Strategic Technology Trends
Source: http://www.gartner.com/technology/research/top-10-technology-trends/
![Page 6: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/6.jpg)
Tablets
Android63%
iOS32%
Windows Phone4%
Android iOS Windows PhoneWindows RT Others
Smartphone
Android79%
iOS13%
Windows Phone4%
BlackBerry3%
Linux1%
Android iOS Windows PhoneBlackBerry Linux SymbianOthers
Mobile Market Share
Source: http://www.idc.com/getdoc.jsp?containerId=prUS24253413
Source: http://www.idc.com/getdoc.jsp?containerId=prUS24257413
![Page 7: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/7.jpg)
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
![Page 8: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/8.jpg)
![Page 9: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/9.jpg)
Waterfall approach for testing
• Party• Project Definition• Project Realization• Testing (used a project buffer)• Deployment to production• Party
Result: low return on investment on testing effort
![Page 10: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/10.jpg)
Test Case Implementation & Execution – ROI?• Reusablity across
• Different mobile OS versions (new)• Different mobile OS vendors (new)• Different hardware (new)• Different versions of the app under test• Different browser
• Independent of• Screen resolution• Keyboards• Localization
• Re-use of existing• Testing tools• Automation expertise• Scripting / Programming language
![Page 11: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/11.jpg)
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
![Page 12: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/12.jpg)
Manual Testing
• Not reproducible• Time consuming• On Premise?• Test Coverage
![Page 13: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/13.jpg)
Jamo Solutions - Mission
The mission of Jamo Solutions is to provide technology and expertise that enables successful testing and monitoring solutions around the mobile end user experience.
![Page 14: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/14.jpg)
Object Attributes & Identification
• Objects are identified by a unique combination of attributes
• This combination can be set seperately for every object or globaly for all objects of the same type
• Access to all attributes of an object
• Key advantage of object recognition based approach
• Depending on the types of application you are creating — mobile browser, native, hybrid and the level of interactivity (use of gesture sets, access to platform features), this lower-level access will be the stronger path.
Gartner, Magic Quadrant for Integrated Software Suites, 11 July 2013
![Page 15: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/15.jpg)
A typical UI
Consider the differences between testing this application using OCR (screen scraping/bitmap recognition) and object recognition based approach
• All buttons share the same label• All buttons share the same form• All buttons share the same size
• Object recognition: direct access• Reliable• Reusable
• OCR: Need for more intelligence• Error prone• Reduced reusability
![Page 16: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/16.jpg)
Portrait view vs. landscape view
![Page 17: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/17.jpg)
Native, Web & Hybrid Applications
• Native app: packaged application deployed to the phone
• Web app: accessed through the browser
• Hybrid app: native app with embedded web views
Native App• Java • C# • C++• Objective C• ....
Web App• HTML• CSS• Java Script• Runs inside a
browser
Hybrid App
![Page 18: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/18.jpg)
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
![Page 19: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/19.jpg)
Functional Testing - Overview
Mobile Test Automation• M-eux Test allows you to automate your mobile testing
scenarios• Based on future-proof, object recognition based
technology (no jail breaking / rooting, no OCR, iOS7 compatible)
• Wide range of support across mobile platforms (iOS, Android, Windows, BlackBerry)
![Page 20: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/20.jpg)
Demo
![Page 21: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/21.jpg)
Functional Testing - Overview
Integration with exiting tools & processes• Integration with industry-standard development tools
(HP UTP/QTP, Microsoft,...)• Integration with application life cycle management tools
(HP ALM/QC, Microsoft, IBM RQM, ...)• Allows effective re-use of and integration with your
existing testing processes and tools• Minimizes the total investment, increases the ROI
![Page 22: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/22.jpg)
Browser-Based Functional Testing
What?• Allows you to automate the testing of web applications on mobile platforms
• Based on our proven object recognition based technology
• Allows for seamless cross-browser, cross-platform, cross-device scripting
• You test the real app; on real devices or simulators/emulators
![Page 23: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/23.jpg)
Demo
![Page 24: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/24.jpg)
Browser-Based Functional Testing
Cross Browser Script
![Page 25: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/25.jpg)
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
![Page 26: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/26.jpg)
26
Performance/Capacity Management
Aspects of performance/capacity management• Performance Testing
• Load Generation• Response Time Measurement
• Performance Monitoring
04/11/2023 © Jamo Solutions
![Page 27: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/27.jpg)
27
Performance/Capacity Management
Load Generation• M-eux Test allows you to create automated test cases• You can replay these devices on multiple devices
simultaneously (physical or virtual)• We integrate with load generation tools (LoadRunner,
VS,…) through UFT/VS integration
04/11/2023 © Jamo Solutions
![Page 28: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/28.jpg)
28
Performance/Capacity Management
Response Time Measurement• M-eux Test allows you to create scripts that execute an
action and wait for the operation to complete , and report timing data
• You can integrate this data with ALM tools (HP, Microsoft,…) to track performance over time
04/11/2023 © Jamo Solutions
![Page 29: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/29.jpg)
29
Performance/Capacity Management
Performance Monitoring• M-eux Test allows you to create scripts that execute an
action and wait for the operation to complete or fail, and report timing data
• You can integrate this data feed with your NOC operational tools (HP, Microsoft,…)
04/11/2023 © Jamo Solutions
![Page 30: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/30.jpg)
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
![Page 31: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/31.jpg)
31
Best Practices Management
• Re-use scripting best practice from PC world:• Descriptive programming• Exception handling• Object reconfiguration• Frameworks ( Linear framework, function decomposion
framework, keyword-driven framework, data-driven framework, etc.)
• Best pracitices on setup• Lab setup• Scripting setup
![Page 32: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/32.jpg)
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
![Page 33: Mobile Application Testing af Frederik Carlier, Jamo Solutions](https://reader031.vdocuments.us/reader031/viewer/2022012922/554d309db4c905c5208b55a5/html5/thumbnails/33.jpg)
Conclusion
• Let your test be reproducable• Manage frequent test cycles• Create re-usable tests• Object recognition is the way to go• Change your system and app as less as possbile by
instrumentation• Note that the app is not the only thing to test