bugdebug-mobility testing (calsoft labs)

8/2/2019 BugDebug-Mobility Testing (Calsoft Labs)

http://slidepdf.com/reader/full/bugdebug-mobility-testing-calsoft-labs 1/20

Future of Testing – Impact of Mobile DevicesSomenath Nag

24th March, 2012Copyright © 2011 Calsoft Labs



Copyright © 2011 Calsoft Labs

Future of Testing – Impact of Mobile Devices

Over last couple of years, mobile devices have shown a

phenomenal growth, at the same time PC industry is on a declining

path. Due to this, we are experiencing a paradigm shift on howapplications are built, tested, and used by the end users, and has a

potential to create a disruption in the traditional way of software

development and QA. Though it started with consumers,

enterprises are also embracing mobility more and more, especially

after the stupendous success of iPad. At the same time enterprises

are also facing challenges in the area of provisioning, data

management, device management, and security. Mobile devices

are also used differently compared to traditional computing

platforms. Due to this mobility devices and platforms throw up a

new kind of challenges to testing fraternity. Calsoft Labs, with its

unique competence and experience of working with leading

Software and Hardware companies, has been in the forefront of

mobility. Calsoft Labs’ Mobility & Testing practices have beenworking together for some time to build methodologies, processes,

and frameworks to address the challenges arising because of the

above mentioned scenarios and to create a new paradigm in

application and product testing.

Somenath Nag

Director - Business

Development & Marketing,

Calsoft Labs

http://in.linkedin.com/in/somenathnag






Content

• How is Mobility Changing the World

• Market Opportunity

• Challenges in Testing Mobility Applications

• Addressing the Challenges




Can we think of a day in our life without cellphone?

COMMUNICATION

ENTERTAINMENT

U T I L I T I E S /

T O O L S P

R OD

U C T I V I T Y




Same is true for enterprises also

Task and Business Users

Line of Business

Managers

Consumers and Ecosystem

Execs

Mobile E-Commerce Mobile Marketing Self-Service




When every industry is worried about recession,

mobility is growing at a supersonic speed

585,000+ Apps

25,000,000,000+ Downloads

400,000+ Apps

10,000,000,000+ Downloads




The unique challenges in testing mobile applications

Diversity in Device Management

Application Usage

UX Design

Network consideration

Application & Device security

Rapid Application Testing




Challenge 1 - Diversity in Device Management

• Different application platforms

– Each platform has its own standards, application interface, Programminglanguages

– Needs to test applications on each platform for each features

– Different certification process for different platforms

– Increases time, cost, and effort

• Device factors

– No standardization of device & form factors

– New innovations creating more complexities

– Following is a list of the factors that can affect your application behavior:

• Screen size• Android OS versions

• CPU




Challenge 2 - Paradigm Shift in Application Usage

• Indoor use

• Application dependent

• Data entry through key-board

• Low or zero interruptions

• Low environmental input

• Information Processing through

normal funnel

• Anytime, Anywhere

• Context Sensitive

• Complex data entry process (key-pad, touch, voice, etc.)

• High interruptions

• High environmental input

(Camera, voice, etc.)

• Information processing through

an inverted funnel

PC Application Mobile Application




Challenge 3 – Evolving Usage Patterns and innovations

are creating new testing challenges

• No established mobile software usability culture –

different for different platforms

• Direct usage of usability principles used in PC based

applications

• Impact of smaller form factors on usability• Handling interruptions

• Interaction with other native and third party

applications (camera, voice, music players)

• Complexity created by gestures, multi-touch and

screen rotation




Challenge 4 - Network Considerations

• Unlike the network landscape of the PC

environment, the network landscape of a mobiledevice may have gateways

• Applications should be tested for their compatibility

with WAP-enabled as well as HTTP-enabled devices

• Unpredictable network latency leading to

inconsistent data transfer speeds.

• Gateways in a wireless network may act as data

optimizers that may result in decreasedperformance for heavy traffic




Challenge 5 - Application and Device Security Threat

Model

Courtesy: OWASP Project


Ch ll 5 A li i d D i S i Ri k h



Copyright © 2011 Calsoft Labs14

Challenge 5 - Application and Device Security Risks that

Need to be Tested

•

Insecure Data Storage• Improper Session Handling

• Weak Server Side Controls

• Security Decisions Via Untrusted Inputs

• Insufficient Transport Layer Protection

• Side Channel Data Leakage

• Client Side Injection

• Broken Cryptography• Poor Authorization and Authentication

• Sensitive Information Disclosure

Top 10 Security Risks


Courtesy:

OWASP Project




Challenge 6 – Rapid Application Testing

• In order to deliver the benefits of faster time to

market, RAD environments are used for mobileapplication development.

• Since the time taken for development is reduced by

the introduction of RAD tools, builds will be

available for testing much earlier

• RAD methodology imposes an indirect pressure on

testing teams to reduce the testing cycle time

without compromising quality and coverage• This pressure becomes more when applications are

targeted for multiple platforms simultaneously

b l k f dd




A Mobility Testing Strategy Framework for addressing

these challenges

Test

Strategy

Network

Landscape

Device

Landscape

Usage

Scenarios

Performance

Benchmark

Application

Category

Security

Consideration

(WOASP)

Automation Consideration

Data Centric Apps

Device Capability

based AppsO

P

Test Execution

• Use a Weighted Device Platform Matrix method to identify the most critical

hardware/ platform combination to test

• Check the end-to-end functional flow in all possible platforms at least once.

• Conduct performance testing, GUI testing, and compatibility testing using actual

devices.

• Measure performance only in realistic conditions of wireless traffic and user load.

• Select the right automation test tool




Suggested Guideline for Tool Selection

• Support for ALL object identification methods

• Secure solution

• Ability to test devices taken out of a retail store - No

Jailbreak/Rooting

• Support for all device functionalities such asgestures, security alerts, wake and reboot

• Plugs-in to existing test environment

• Same test should run on different devices




Q&A

Thank You




About Calsoft Labs & Alten

• Embedded Product

Engineering Service

• Software Product

Engineering Service

• Independent Verification

& Validation Service

• Design Engineering

Service

• Telecom

• Healthcare

• Education

• Enterprise Software

• Automotive

• Media & Entertainment

• Semiconductor

• Aeronautics / space• Financial

• 15000+ people

• 65% of engineering team

with 5 years+ experience

• 27% Master Degree

holders

• ALTEN is one of Europe's

most dynamic employers,

hiring almost 2,000 newpeople each year.

• Located in 16 countries

• 10 Delivery centers

• Design & Development

Centers in Bangalore,

Mysore & Chennai

• US Delivery Centre in

Boston, MA

Service Offerings Industry Focus People Locations

Our Partners

Calsoft Labs

■ Wholly owned subsidiary of ALTEN

■ Outsourced Product Design arm of the ALTENGroup

■ Pioneer in Product Engineering Services

■ Over 300 satisfied customers

■ Over 1000 products developed and tested

ALTEN SA

■ Established in 1988

■ European Leader in Engineering andTechnology Consulting services

■ 14800 employees worldwide, >90%

engineers and consultants

■ Located in 16 countries

■ Revenue of € 1 Billion+ for FY 2011




Calsoft Labs’ Testing Practice

• Test Strategy Consulting

• Embedded Product

Testing

• Software Product Testing

• Test Automation

• ‘On-Demand’ Testing

• Compliance testing• Security Testing

• Telecom

• Healthcare

• Education

• Enterprise Software

• Automotive

• Media & Entertainment

• Semiconductor• Aeronautics / space

• Financial

• OWASP Compliance

• Accelerator for OWASP

Compliance

• Home-made Open Source

Automation Framework

• Field testing and

Interoperability

• Carrier validation test

• Protocol and

Conformance test

• Device Management

Testing• Mobile App testing

• Usability Testing

Service Offerings Industry Focus Thought Leadership Mobility Testing

■ 200+ dedicated test professionals

■ Track record

■ QA solutions throughout the SDLC based on TMMi framework

■ Reusable repository of test cases, scenarios, and validation libraries

Partner

bugdebug-mobility testing (calsoft labs)

Documents