non-functional testing on mobile...
TRANSCRIPT
![Page 1: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/1.jpg)
Non-Functional Testing on
Mobile Devices Nikolai Pavlov
![Page 2: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/2.jpg)
Slide 1
Outline
1. Introduction
2. Functional Requirements
3. Non-Functional Requirements
4. Non-Functional Testing: KPIs
5. Summary
![Page 3: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/3.jpg)
Slide 2
Introduction
Nikolai Pavlov
• 2005-2007 Mobile QA Engineer / Skype
• 2007-2008 Technical PM in Mobile / Skype
• 2008-2011 Mobile QE Manager / Skype
• 2011-… Skype Mobile QE Lead / Microsoft
• 2007 TTU, Computer Systems, B.Sc.
• 2011 TU, IT management, MBA
![Page 4: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/4.jpg)
Slide
Requirements
Functional vs. Non-functional
WHAT? vs. HOW?
![Page 5: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/5.jpg)
Slide
Functional Requirements
Use Case
BUSINESS
Functional
Requirement Test Case
synergy
![Page 6: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/6.jpg)
Slide
Non-Functional
Requirements Non-Functional requirements categories:
• Performance
• Security
• Usability
• Localization
• Scalability
• etc….
![Page 7: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/7.jpg)
Slide
Non-Functional
Requirements Functional vs. Non-functional
User MUST be able to sign in Sign-in time should be equal
or less than 5 seconds
User MUST be able to switch
application between foreground
and background
Maximum delay in toggling
foreground / background is
less than 1 second
![Page 8: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/8.jpg)
Slide
Non-Functional
Requirements
• What if we can’t meet the requirement?
• How far from meeting the requirement?
• Are we making the right changes?
• …
![Page 9: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/9.jpg)
Slide
Non-Functional
Requirements: KPIs
WHAT?
1 2 3
HOW? DATA
![Page 10: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/10.jpg)
Slide
Non-Functional
Requirements: KPIs
5. Battery Life
1.Application Size 2. Start-Up time 3. Responsiveness
4. Memory Footprint
![Page 11: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/11.jpg)
Slide
Application Size: How?
![Page 12: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/12.jpg)
Slide
Application Size: Why?
Monitor
1
Debug
2
Optimize
3
![Page 13: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/13.jpg)
Slide
Start-Up: Importance
1. Mobile Users are always in a hurry
2. Competitors
3. Customers
![Page 14: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/14.jpg)
Slide
Start-Up: How?
1. Integrate into your
automation tests
2. Use Test Frameworks
3. Use Record-Play
![Page 15: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/15.jpg)
Slide
Responsiveness: Importance
1. Mobile Users are always in a hurry
2. Competitors
3. Customers
![Page 16: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/16.jpg)
Slide
Responsiveness: How?
1. Integrate into your
automation tests
2. Use Test Frameworks
3. Use Record-Play
![Page 17: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/17.jpg)
Slide
Memory Footprint: How?
1. Define test suite
2. Define baseline
3. Manual – first, ultimate goal - automate
![Page 18: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/18.jpg)
Slide
Memory Footprint: How?
![Page 19: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/19.jpg)
Slide
Memory Footprint: How?
WP Performance
Analysis tool
Device Status for
Windows Phone
DDMS
android.os.Debug
xcode Instruments
![Page 20: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/20.jpg)
Slide
Battery Consumption: Why?
• Provide visibility to battery consumption
changes with every new integration
• Provide battery consumption data to main
stakeholders and partners
• Provide battery consumption data for making
decisions
![Page 21: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/21.jpg)
Slide
Battery Consumption: How?
![Page 22: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/22.jpg)
Slide
Battery Consumption: How?
![Page 23: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/23.jpg)
Slide
Battery Consumption: How?
![Page 24: Non-Functional Testing on Mobile Devicesnordictestingdays.eu/files/files/non-functional_testing_on_mobile... · • 2005-2007 Mobile QA Engineer / Skype • 2007-2008 Technical PM](https://reader034.vdocuments.us/reader034/viewer/2022050120/5f5055335d86b4080243d2dc/html5/thumbnails/24.jpg)
Questions