shift left mobile application testing
Post on 10-Feb-2017
135 Views
Preview:
TRANSCRIPT
W14 Track Session 4/20/2016 2:00 PM
"Shift Left Mobile Application Testing"
Presented by: Satyajit Malugu
GoDaddy
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ info@techwell.com ∙ www.techwell.com
Satyajit Malugu GoDaddy A senior SDET at GoDaddy, Satyajit Malugu is focused on mobile testing. His work involves automation of native and hybrid apps, and providing test perspective to his team that recently converted from waterfall to agile. As a testing leader in a company that is deploying a suite of native apps, Satyajit is involved with strategizing, executing, and mentoring other testers and stakeholders on best practices for native mobile testing. Previously, Satyajit worked at Urbanspoon and gained SDET knowledge at Microsoft. He frequently presents on mobile topics. Read his blog and follow him on Twitter at @malugu.
Shift left Mobile Application TestingSatyajit Malugu
Urbanspoon
Specification Design Development Maintenance
Domain Finder
Specification Design Development Maintenance
Still had lot of problems
• Testing unable to fit into agile dev cycle
• UI automation unable to keep up with constant changes
• A very low coverage after weeks of playing catchup
Test pyramid reversed
http://adventuresinqa.com/2015/10/19/mobile-test-pyramid/#more-3194
What is shift left
• Testing processes start along with development processes
Godaddy Investor
Specification Design Development Maintenance
Why Shift left
• Cost of fixing bugs exponential across SDLC
Specification Design Development Maintainance
Enabling Agile
• Agile is all about adaptability to change
• Enabled through fast feedback loops
• Test early and often
Challenges for shift left in mobile
• Tool support not mature
• Device fragmentation
• Culture of native development teams
Tool support is getting better
• Companies dedicated to mobile testing
• Real device clouds
• Network variations
• Sensor simulation
Android tool support
• Unit tests – JUNIT
• Code coverage – Jacaco
• Continuous integration – Jenkins
• Functional tests – Robotium, roboelectric, Espresso etc
• Continuous deployment – Alpha, beta releases in play store
• Device emulation – adb, genymotion
• Performance testing – rendering profiler, traceview
iOS tool support
• Unit tests – XCTest,
• Code coverage – Xcode server with bots
• Continuous integration – Jenkins/Xcode server
• Functional tests – EarlGrey
• Continuous deployment – Testflight alpha, beta
• Device emulation – Xcode
• Performance testing – XCTest, instruments
Reclaiming the pyramid
Step I – Measure unit test code coverage
• What you can’t measure, can’t be improved
• Get the team to agree on a good coverage %
• Send coverage report everyday
Slack bots reporting daily coverage
Step II– Create a CI infrastructure
• For running unit, functional, UI tests
• Build various environments
• Code coverage reports
Pull Request triggers unit and UI tests
Jenkins UI test run sample
Step III– Replace some UI tests with unit tests
• As a principle prefer lower level test over a higher one
• Don’t repeat a test - DRY
Step IV- Partial integration testing
• Create test hooks for navigating into app directly
• Deep links, shortcuts
• Level – 1 verifications
• Not everything need to be verified through UI
Step V- Re-architech E2E tests
• Should be minimal• Run them continuously• On a cloud provider
• Should be atomic not interdependent• Enables running in parallel
• A single test can be triggered in multiple ways• Different language
• Different screensize
• Different OS
Specification Design Development Maintenance
1. Continuous testing2. Crash Monitoring3. Performance Monitoring4. App store reviews monitoring5. Engagement monitoring
1. Functional tests2. UI Tests
1. Testability hooks2. Accessibility ids3. API level testing
1. Test infrastructure2. CI/CD setup3. Device lab
Summary
On the horizon – React Native
• Bringing web like deployment capabilities to native world
• Learn once, write anywhere
• Very high importance to testing
top related