automated concolic testing of smartphone apps
DESCRIPTION
Automated Concolic Testing of Smartphone Apps. Saswat Anand Stanford Univ . Mayur Naik Georgia Tech . Hongseok Yang Univ. of Oxford. Mary Jean Harrold Georgia Tech. Motivation. Motivation. Problems with Smartphone Apps. Problem. - PowerPoint PPT PresentationTRANSCRIPT
Automated Concolic Testing
of Smartphone Apps
Saswat AnandStanford Univ.
Mayur NaikGeorgia Tech.
Hongseok YangUniv. of Oxford
Mary Jean HarroldGeorgia Tech.
Motivation
Motivation
Problems with Smartphone Apps
Automatically generate test inputs for bounded exhaustive testing of
smartphone apps
Problem
Test Inputs for Apps• Whole-program testing• Test input is a sequence of events e1, e2…,
en
• Types of events: a tap on the screen, change in geo-location, arrival of a SMS message, etc.
Bounded Exhaustive Testing of Apps
S, the set of all event sequences* s.t. each sequence
takes a unique path
*of bounded-length
Set of covered
branchesGoal: cover
these
1. Generate individual events2. Generate sequences of events
Two subproblems
Generating Individual Events• An event is associated with data
o X & Y coordinates of a tap evento geo-location of a change-in-geo-location evento content of an incoming SMS evento etc.
• Data determine which program path is taken
Challenge: Generate the “right” data for events
Example: Music Player App
Play Pause
Stop Eject
Rewind Skip
Example: Music Player App
public void onClick(View target) { if (target == play) startService(new Intent(ACTION_PLAY)); else if (target == pause) startService(new Intent(ACTION_PAUSE)); else if (target == skip) startService(new Intent(ACTION_SKIP)); else if (target == rewind) startService(new Intent(ACTION_REWIND)); else if (target == stop) startService(new Intent(ACTION_STOP)); else if (target == eject) showUrlDialog();}
tap(136, 351)
Example: Music Player App
public void onClick(View target) { if (target == play) startService(new Intent(ACTION_PLAY)); else if (target == pause) startService(new Intent(ACTION_PAUSE)); else if (target == skip) startService(new Intent(ACTION_SKIP)); else if (target == rewind) startService(new Intent(ACTION_REWIND)); else if (target == stop) startService(new Intent(ACTION_STOP)); else if (target == eject) showUrlDialog();}
tap(248, 351)
Example: Music Player App
public void onClick(View target) { if (target == play) startService(new Intent(ACTION_PLAY)); else if (target == pause) startService(new Intent(ACTION_PAUSE)); else if (target == skip) startService(new Intent(ACTION_SKIP)); else if (target == rewind) startService(new Intent(ACTION_REWIND)); else if (target == stop) startService(new Intent(ACTION_STOP)); else if (target == eject) showUrlDialog();}
tap(360, 351)
Example: Music Player App
public void onClick(View target) { if (target == play) startService(new Intent(ACTION_PLAY)); else if (target == pause) startService(new Intent(ACTION_PAUSE)); else if (target == skip) startService(new Intent(ACTION_SKIP)); else if (target == rewind) startService(new Intent(ACTION_REWIND)); else if (target == stop) startService(new Intent(ACTION_STOP)); else if (target == eject) showUrlDialog();}
tap(24, 351)
Example: Music Player App
public void onClick(View target) { if (target == play) startService(new Intent(ACTION_PLAY)); else if (target == pause) startService(new Intent(ACTION_PAUSE)); else if (target == skip) startService(new Intent(ACTION_SKIP)); else if (target == rewind) startService(new Intent(ACTION_REWIND)); else if (target == stop) startService(new Intent(ACTION_STOP)); else if (target == eject) showUrlDialog();}tap(136, 493)
Example: Music Player App
public void onClick(View target) { if (target == play) startService(new Intent(ACTION_PLAY)); else if (target == pause) startService(new Intent(ACTION_PAUSE)); else if (target == skip) startService(new Intent(ACTION_SKIP)); else if (target == rewind) startService(new Intent(ACTION_REWIND)); else if (target == stop) startService(new Intent(ACTION_STOP)); else if (target == eject) showUrlDialog();}tap(305, 544)
Existing alternatives• Random Testing
oCannot perform systematic/exhaustive testing
• Platform-specific tools (e.g., hierarchy viewer in Android)oLimited to GUI EventsoCannot handle third-party GUI widgets
Generating Individual Events
Generating Individual Events
Our solutionUse concolic execution to generate data
associated with events
FT
FT
tap(int x, int y){1 if (x>2 && x<4){2 if (y>1 && y<3)3 W1_clicked();4 else5 W2_clicked();6 }else7 W3_clicked(); }
Generating Individual Tap Events
1
72
3 5
x>2 && x<4
y>1 && y<3
Generating Individual Tap Events
tap(1, 5)
FT
FT1
72
3 5
x>2 && x<4
y>1 && y<3
Generating Individual Tap Events
tap(1, 5)
F1 !(x>2 && x<4)
W3_clicked()
FT
FT1
72
3 5
x>2 && x<4
y>1 && y<3
(x>2 && x<4)
tap(3, 5)
(x>2 && x<4)
Generating Individual Tap Events
tap(1, 5)
T1 (x>2 && x<4)F2 !(y>1 && y<3) W2_clicked()
tap(3, 5)
FT
FT1
72
3 5
x>2 && x<4
y>1 && y<3
(x>2 && x<4)(y>1 && y<3)
tap(3, 2)
Generating Individual Tap Events
tap(1, 5)
(x>2 && x<4)(y>1 && y<3)
tap(3, 5) tap(3, 2)
T1 (x>2 && x<4)T2 (y>1 && y<3) W1_clicked()
FT
FT1
72
3 5
x>2 && x<4
y>1 && y<3
Example: Music Player App
❖
❖
❖
❖ ❖❖
❖ ❖
❖❖ ❖
1. Generate individual events2. Generate sequences of events
Two subproblems
Generating Sequences of EventsConcatenate individual events generated by concolic execution.
Baseline Algorithm
Set of covered
branches
S, Set of all event sequences s.t. each sequence takes a
unique pathBaseline
algorithm
Goal: cover these
Number of sequences generated for Music Player app by baseline algorithm
Baseline Algorithm Suffers from Path Explosion
1 2 3 40
5000
10000
15000
20000
25000
ACTEve Algorithm
ACTEve: Automated Concolic Testing of Event-driven programs
ACTEve Algorithm
Set of covered
branches
R s.t. R S⊆
S, Set of all event sequences s.t. each sequence takes a
unique pathBaseline
algorithm
ACTEve algorithm
Goal: cover these
ACTEve is relatively sound
Path Subsumption
¿ γ ,C>¿Maps memory location to values (symbolic or concrete)
Path constraint
Program state in concolic execution
Program entry
Path Path
¿ γ1 ,𝐶1>¿ ¿ γ2 ,𝐶2>¿
subsumes
Path SubsumptionNote - memory map – path constraint
Program entry
Path Path
¿ γ1 ,𝐶1>¿ ¿ γ2 ,𝐶2>¿
subsumes
Path SubsumptionNote - memory map – path constraint
- Don’t generate test corresponding to any path that is an extension of - Only generate tests corresponding to paths that are extension of
• Checking path subsumption is very expensive in generalo Constraint implication checkoMatching memory map
• But, path subsumption can be checked cheaply in special caseso Read-only eventso Events whose mutual ordering does not mattero etc.
Path Subsumption
event is does not write to any memory location.
Read-only EventsProgram Entry
corresponds to
corresponds to
Path executed for event sequence
is subsumed by q
❖
❖
❖
❖❖ ❖❖❖
❖ ❖❖
Read-only Events
Read-only events are represented as ❖
ACTEve System Architecture
Empirical Study
• Apply ACTEve and baseline algorithmso event sequences of length up to 4o 16 concurrently running emulators o time budget of 12 hours
• Measured three metricso running timeo number of feasible pathso number of satisfiability checks
Empirical Results
Future Work
Widget Explosion
1. Concolic execution to generate individual events
2. ACTEve: an efficient algorithm for bounded exhaustive testing of event-driven programs o Requires only a small fraction (5-36%) of time
compared to baseline algorithm
3. Implementation for Android
Main Contributions
Backup slides
¿ γ 1 ,𝐶1>¿
¿ γ 2 ,𝐶1∧𝐶 ′>¿
1. because does not write to any memory location.
Read-only EventsProgram Entry
corresponds to event sequence
corresponds to in
Path executed for input event sequence
Output of Android’s “Hierarchy Viewer” tool
A Solution: Use Platform-specific Knowlege
A Solution: Use Platform-specific Knowlege
void onTouchEvent(MotionEvent e) { int rawX = (int) e.getX(); int rawY = (int) e.getY(); int x = (rawX – MARGIN) / SIZE; int y = (rawY – MARGIN) / SIZE; if (x >= 0 && x < 3 && y >= 0 & y < 3) { int cell = x + 3 * y; … }
Output of Android’s “Hierarchy Viewer” tool
Program Entry
Path
{is feasible} {is feasible}
Path
Program Entry
Covered branches
Covered branches
same program location
Path Subsumption
Program Entry
Path
{is feasible} {is feasible}
Path
Program Entry
Covered branches
Covered branches
same program location
if we explore all paths that extends , then no need to explore any path that extends because no additional branch coverage will be obtained.
Path Subsumption
Path constraint when PAUSE button is tapped on
Example: Music Player App