liuppa – movies team dsm’10 – reno/tahoe, nevada, usa 17 october 2010 a dsml for mobile...

LIUPPA – Movies TeamLIUPPA – Movies Team

DSM’10 – Reno/Tahoe, Nevada, USA 17 October 2010

A DSML for Mobile Applications Testing

Youssef RIDENE*/**, Nicolas BELLOIR*, Franck BARBIER* and Nadine COUTURE***

* Liuppa Labs, Movies Team, University of Pau, France** Neomades Cie, Bidart, France

*** Estia, Bidart, France

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 2

Agenda

Problem & Motivation

Cloud Testing

The DSML

Conclusion

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 3

Problem & Motivation

Mobile Application Development Ecosystem’s

Smartphones: iPhoneAndroid BadaWindows Phone…

Mass market: Java ME

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 4

Problem & Motivation

Smartphones:Development environmentProgrammaing languages, APIs…

Java ME:Standards: MIDP (1.0, 2.0, 2.1), CLDC (1.0, 1.1)…Optional APIs: File IO, PIM, Bluetooth…Implementation diversity: different interpretation, bugs, security policySpecific hardware (screen size, inputs methods…)

How many specific versions of each application? (Porting Tools!)

How to ensure end-user quality assurance?

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

5

Problem & Motivation

Application tests are made on real handsets ~ 250 in average

Visual and sonorous interpretation of the application’s behavior

Costly:TimeHandsetsTesters’ TeamFrom scratch for each application

DSM‘10

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

6

Problem & Motivation - Test steps sample

Alternatives?

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

Alternative

Provide more efficient environment

Manage variability

LMO'07 7

Cloud Testing PlatformCloud Testing Platform

DSMLDSML

DSMLDSML

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 8

Agenda

Problem & Motivation

Cloud Testing

The DSML

Conclusion

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 9

Cloud Testing - Architecture

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 10

Cloud Testing – the Neomades prototype

Internal prototype for our research

?

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 11

Cloud Testing – Other solutions

Device Anywhere

Perfecto Mobile

Keynote

…

How to manage variability?

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 12

Agenda

Problem & Motivation

Cloud Testing

The DSML

Conclusion

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 13

The DSML - Description

A Modeling Language to design test scenarios

Resembling UML Sequence diagrams Lifelines: Tester or MobilesMessages:

Tester Mobile: press a key, press the pointer…Mobile Mobile: call, send SMS…

No programming skills are required

A scenario can be adapted for another application

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 14

The DSML - Description

Basic domain elementsTester Mobile under test A second Mobile (optional)Simple actions: press a key…

Variabiliy ManagementInterruptionsAutomates actionsCheckPoints and results validation

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 15

The DSML

A scenario is composed by a tester, one or more mobiles

A scenario is composed by a tester, one or more mobiles

A mobile can permorm actionsA mobile can permorm actions

A tester can execute interactions on the mobile, called messages

A tester can execute interactions on the mobile, called messages

User can defines checkpointsUser can defines checkpoints

Three variability types can be definedThree variability types can be defined

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 16

The DSML - Basic domain elements

Tester Mobile under test A second Mobile (optional)

Simple actions: press a keypress pointerrotate the phone…

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 17

The DSML - Tool

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 18

The DSML - Automated actions

Write TextSend SMSSet Wifi connectionSet date and Time…

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 19

The DSML - Interruptions

Test application behaviorSend SMSBluetooth connection

Incoming call …

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 20

The DSML - Variability Management

The same application can vary depending on: Different mobile screen resolutions Mobile models: Nokia N95, Samsung Galaxy S… Mobile specifications: has GPS, has Bluetooth…

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 21

The DSML - Variability Management

In practice: Create a variability point Assign one ore more action to this variability point

A scenario can contains many variability points

The execution paths are managed on the testing bed thanks to a database (which phone has GPS…)

Actions will be executed only on the targeted phones.Only one scenario for all the devices

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 22

The DSML - Example : Mobile specifications

Actions assigned to touch screen enabled phones

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 23

The DSML - Example : Screen Size

Actions performed only on mobile phones with a 240x320 resolution

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 24

The DSML - Example : Mobile Models

Assign actions only to one or more specific device

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 25

The DSML - Check Points

Take a SnapshotRecord an Audio SequenceRecord a video sequence

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 26

The DSML - Result validation

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 27

The DSML - Tools

Eclipse Modeling Framework Metamodel of the DSL

Graphical Modeling Framework Modeler + Eclipse environment (drag & drap, copy/paste, appearance…) - Bugs!

Eclipse RCP and plugins Link to the testing bed for scenario execution

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 28

Agenda

Problem & Motivation

Cloud Testing

The DSML

Conclusion

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 29

Conclusion

Develop a cloud testing platform A potential collaboration with existing test beds

Created a DSML To specify test scenarios Manage Variability

Evaluation : Adoption by testers

LIU

PP

A –

Movie

s T

eam

LIU

PP

A –

Movie

s T

eam

DSM‘10 30

Perspectives

An iterative testing process with a specialized team to enhance our tool

Emerging platforms (Tablets…)

Control the applications @runtime thanks to Autonomic Computing

A component-based programming approach to a better description of the application’s flowchart

LIUPPA – Movies TeamLIUPPA – Movies Team

Thank you!

Questions, please?

nicolas.belloir@univ-pau.fr

DSM’10 – Reno/Tahoe, Nevada, USA October 2010, 17th