a guided approach to testing - a mobile first strategy for your enterprise!
DESCRIPTION
Check out this presentation given by Rutesh Shah, CEO of InfoStretch Corporation at Enterprise Application Revolution.TRANSCRIPT
©2014 InfoStretch Corporation. All rights reserved.
A Guided Approach to Testing - A Mobile First Strategy for your Enterprise
©2014 InfoStretch Corporation. All rights reserved.
InfoStretch Overview Enable Customers to Monetize Mobile Channel
Helping you Achieve Enterprise-Grade Mobile Applications
2
©2014 InfoStretch Corporation. All rights reserved.
The Company
Unique Apps Tested
Employees
US, UK, India
150+ Apps developed and deployed for Enterprises
Real devices in mobile labs 3000+
Global Footprint
Mission
Locations Corporate HQ: Silicon Valley
Offices: Conshohocken (PA), Ahmedabad(India), Pune(India), London(UK)
500+
200K
Active Mobile Customer Engagements 60+
To Create Meaningful Customer Engagement and Improve
Employee Enablement and Productivity with Mobile Applications
3
©2014 InfoStretch Corporation. All rights reserved.
App Store Certification Authority (2007 onwards)
4
Tested 200,000+ mobile applications in more than 16
languages and 20+ carrier networks around the globe
Serving 4 Largest Mobile App Stores for Certification
©2014 InfoStretch Corporation. All rights reserved.
InfoStretch – 2013 Onwards
Deploy Deep
Mobile Skills
Design, UI &
Development for
Enterprise
Applications
Enterprise
Testing and
Automation with
Mobile Center of
Excellence(mCOE)
Optimization and
Post Production
Sustenance
Unified Platform for
Building Mobile
Solutions
Enterprise Mobility
Tools Stack to
Power Service
Engagements
5
©2014 InfoStretch Corporation. All rights reserved.
Test Challenges for Enterprise Mobile Applications
6
©2014 InfoStretch Corporation. All rights reserved.
Quick Poll
• Do you have a separate team for Mobile Testing or You rely on End-Users?
• Do you have a separate test lab with all mobile devices you support?
• Are you familiar with Mobile Testing tools?
7
©2014 InfoStretch Corporation. All rights reserved.
Mobile Testing Challenges
• Proliferation of Devices, Platforms and Carriers
• Compressed Release and Development Cycles
• Generating Real World Testing Conditions
• Limitations of Manual Testing
9
©2014 InfoStretch Corporation. All rights reserved.
#1 Device and Platform Proliferation
DEVICES
CARRIERS
OPERATING SYSTMES
10
©2014 InfoStretch Corporation. All rights reserved.
#2 Compressed Release Cycles
v1.0 v2.0 v3.0 v4.0 v5.0 v2.1
Nov 2013
Mar 2014
Apr 2014 Aug 2014
Dec 2014
Feb 2015
Mobile Applications
Web Applications
v1.0 v2.0 v3.0 v4.0 v5.0 v2.1
Nov 2013 Dec 2014 April 2015 Jan 2016 Sept 2016
Feb 2017
11
©2014 InfoStretch Corporation. All rights reserved.
#3 Generating Real World Conditions
• How will the application behave under 70% CPU utilization?
• How will connection speed change the application’s performance?
• What is the performance impact of 50 other applications running on the device?
• How will it work with interrupts like SMS and MMS?
12
©2014 InfoStretch Corporation. All rights reserved.
#4 Limitations of Manual Testing
• Running the code manually – tedious
• Effort required to run manual test cases – expensive
• Process of manual testing – boring and error prone
• Non repeatability and reuse – time consuming
13
©2014 InfoStretch Corporation. All rights reserved.
Déjà vu
14
©2014 InfoStretch Corporation. All rights reserved.
Need for a Strong Quality Program
Pace of Change
Device and
Platform
Fragmentation
Cost of Failure
Backdoor
Vulnerabilities
Employee
Satisfaction and
Productivity
15
©2014 InfoStretch Corporation. All rights reserved.
Enterprise
Mobile
Infrastructure
Testing
Application
Functionality
Testing
Enterprise Mobility and Quality Needs
Maintenance &
User Support
Testing
©2014 InfoStretch Corporation. All rights reserved.
Quality Strategy Needs For Enterprise MDM
Enroll Configure Secure Service
• Test VPN to enterprise server, Email
• Verify OTA content delivery under varying network conditions
• Verify App Deployment • Verify enforced password
policies • Verify data encryption • Test device compatibility with
anti virus
• Device Analysis • Lab Setup with possible
OEM, OS and Carrier combinations
• Device Compatibility with identified policies
• Test enrollment via SMS, OTA, Email
• Passcode policies • Device Unlock • Containerization/Virtualization • Enforce Encryption • Security updates
• Verify Security updates • Testing data backup • Verify device conformity • Verify Logging/Reading • Verify app-to-app
communication • Verify data leakage
• End of life • Passcode Reset • Tracking Device • Update Software • Device Usage Data • Restore data
• Verify device passcode reset • Verify that device data restore
& remote access • Test end to end SMS/Cloud
Notification • Verify successful app
upgrade/patch to existing application on device
• Verify end of life
• BYOD • Enroll via SMS • Enroll via Email • Enroll via web URL
• Exchange Mail • VPN • Network Settings • Install/configure antivirus
MD
M F
EATU
RE
TEST
STR
ATE
GY
©2014 InfoStretch Corporation. All rights reserved.
QA Lifecycle of Smart Enterprise Apps Infrastructure
Define Test Strategy
•Align QA objectives to Enterprise Mobility Vision
•Define Scope and Limitations of QA
Test Plan
•Device Analysis and Lab Setup
•Data Access and Security
•Testing Containerization
Functional Testing
•App functionality test
•App interoperability test
Device Lifecycle Testing
•Non-functional device specific testing
•Device provision testing
•Device End of life testing
New App Certification Process
•Application Certification on Enterprise App Store
•Enterprise Framework compliance
•Security testing
Ongoing Testing of Service Management
•No harm to data and Networks
•New device validation
•New OS validation
18
©2014 InfoStretch Corporation. All rights reserved.
Types of Mobile Application Testing
Functional
• Validation of Functionality
• Smoke / Regressions Testing
• Off line access testing
• Negative Testing
Non Functional
• Network Strength / Outage / Recovery
• Different NW Types
• Peripheral Testing
Interrupt
• Voice / SMS interrupts
• Notifications
• Battery /Cable Removal
Memory Leak
• Memory Usage
• Memory Leaks
• Garbage Collection
Performance Testing
• CPU Usage testing
• Network Usage
• Page Render time or activity Render time
Usability Testing
• User Experience
• Competitive Analysis
• Expert Review
Installation Testing
• New App Install
• Uninstall and Reinstall
• Upgrade testing
Security Testing
• OWASP Vulnerabilities
• Dynamic Testing
• Static Code Analysis
• Data Encryption
Language Testing
• Validation for Locales
• Images and Text
• Currencies, time zones etc.
• Context
©2014 InfoStretch Corporation. All rights reserved.
Quality – An Integral part of your Enterprise Mobile Strategy
• Comprehensive Testing Infrastructure
• Agile Testing Practices
• Market Change and Application Impact Monitoring
20
©2014 InfoStretch Corporation. All rights reserved.
Enterprise Mobility Quality Enablers
QAS
Selenium
based Mobile
Testing
Solution for
End-to-End
Functionality
Testing
QMetry
A SaaS based
mobile
application
lifecycle
management
and
maintenance
tool
Synchro
Synchronized,
geo distributed
testing on real
devices for
optimal user
experience
management
Attesto
Intelligent UI
heuristics tool
to aggregate
and analyze
user reviews
and perform
verification
and validation
©2014 InfoStretch Corporation. All rights reserved.
UX
Designer
Product
Manager
Developer
Support
Functional
Testing
Business
Use Case
Validation
Synchro
Device
Testing
UI & Layout
Validation
Synchro
Automation
Functional /
Regression
Automation
QAS
Diagnostics
Debug
Issues
Synchro /
QAS
Support
Monitor &
Sustenance
Attesto
Private Device Cloud – 3000+ Devices
Test Management Platform – QMetry /QC
Security
Testing
Load Testing
Monitoring
Non-
Functional
Testing
Retail
Finance
Re
ad
ym
ad
e T
est
Pa
cks
User
Roles
InfoStretch Integrated Solutions – Mobility QA
©2014 InfoStretch Corporation. All rights reserved.
Case in Point Enterprise Mobile MDM QA Strategy for Large Global Insurance company’s Smart App
The Client Profile Company: Large Global Insurance Company
Industry: Insurance and Financial Services
Details:
The Challenge
Main Drivers - Reliability, security and cost
- 70,000 employees around the world
- Mobile applications integrated with other enterprise
apps and field apps
- One of the most aggressive and widespread BYOD program
- Global implementation challenges with wide and diverse
requirements across Geos
- Significant need to use third party certified applications.
- Stringent on field requirements, causing unique networking
challenges
- Validation of the MDM and MAM implementation plan
- Significant post go live changes
©2014 InfoStretch Corporation. All rights reserved.
Phase 1
•Phase 1 – Mobile Testing Strategy (8 weeks)
•Identified overall scope and plan of testing
•Defined multiple aspects of quality to align with MDM and MAM implementation
•Understood current QA team and environment limitations
•Deliverables: Mobile QA plan, high level test strategy for each area (module) of MDM and MAM deployment, project and resource plan, cost estimates
Phase 2
•Phase 2 – Definition of Test lab, Test Areas and Test cases, Testing and Go live (3 months)
•Defined set of devices that meet company requirements and provision them
•Identified functional and non functional tests (300+), relevant test environment and conditions
•Specific test methodology to address unique MAM and MDM implementation testing
•Go live on-time with adequate quality and certification program for external apps
Phase 3
•Phase 3 – Post go live – Special Events Testing (ongoing)
•Post go live test plan
•Identify specific post go live QA trigger points and exit criteria
•Execution of App store certification, Security testing program, specialized events testing
Case in Point Enterprise Mobile MDM QA Strategy for Large Global Insurance company’s Smart App
©2014 InfoStretch Corporation. All rights reserved.
Testing of Event Mgmt. App. for a Large Hi-Tech Company
25
• Event Management Platform Testing
• Self Service – Auto Generated App Testing
• Event Content Testing
©2014 InfoStretch Corporation. All rights reserved.
Thank you ! Email:[email protected]
URL:www.infostretch.com
Blog : blog.infostretch.com