AutoFLox: An Automatic Fault Localizer for Client-Side
JavaScript
Frolin S. Ocariza, Jr.
Karthik Pattabiraman
Ali Mesbah
University of British Columbia
Web 2.0 Applications: JavaScript JavaScript: Implementation of ECMAScript
standard Client-Side JavaScript: used to develop web apps
Executes at client’s browser Web application’s functionality Dependability problem: Average of 4 errors
per web app in the wild [ISSRE11]
2
JavaScript Dependability: Document Object Model (DOM)
3
html
body head
scriptdiv p
Text: “Hello world”
table
tr p
Element added by JavaScript
Element removed by JavaScript
JavaScript Dependability: Document Object Model (DOM)
4
div
id: elem
JavaScript code:
DOM:
var x = document.getElementById(“elem”);var x = document.getElementById(“elme”);
Inexistent IDWill return null
DOM-related JavaScript error
Fault Localization Web application testing
What to do after we find errors? Need to fix them…
Fault localization: Find the root cause of the error
5
Contributions/Outline1. Discussion of JavaScript fault localization
challenges2. Automated technique to localize DOM-
related JavaScript faults3. Empirical studies
Highlight prevalence of DOM-related JavaScript errors
Evaluate accuracy and performance of technique
6
1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer);
3 changeTimer = setTimeout(changeBanner, 5000);
4
5 prefix = “banner_”;
6 currBannerElem = document.getElementById(prefix+currentBannerID);
7 bannerToChange = document.getElementById(prefix + bannerID);
8 currBannerElem.removeClassName(“active”);
9 bannerToChange.addClassName(“active”);
10 currentBannerID = bannerID;
11 }
12 currentBannerID = 1;
13 changeTimer = setTimeout(changeBanner, 5000);
Running Example
7
1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer);
3 changeTimer = setTimeout(changeBanner, 5000);
4
5 prefix = “banner_”;
6 currBannerElem = document.getElementById(prefix+currentBannerID);
7 bannerToChange = document.getElementById(prefix + bannerID);
8 currBannerElem.removeClassName(“active”);
9 bannerToChange.addClassName(“active”);
10 currentBannerID = bannerID;
11 }
12 currentBannerID = 1;
13 changeTimer = setTimeout(changeBanner, 5000);
Goal: Show a fading banner that cycles through four images every 5 seconds
Passed with no argument (even though changeBanner needs one argument)
bannerID will be set randomly
Would return null if bannerID out of range
NULL EXCEPTION!setTimeout call
1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer);
3 changeTimer = setTimeout(changeBanner, 5000);
4
5 prefix = “banner_”;
6 currBannerElem = document.getElementById(prefix+currentBannerID);
7 bannerToChange = document.getElementById(prefix + bannerID);
8 currBannerElem.removeClassName(“active”);
9 bannerToChange.addClassName(“active”);
10 currentBannerID = bannerID;
11 }
12 currentBannerID = 1;
13 changeTimer = setTimeout(changeBanner, 5000);
JavaScript Fault Localization: Challenges Multiple JS execution sequences executing
asynchronously
8
1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer);
3 changeTimer = setTimeout(changeBanner, 5000);
4
5 prefix = “banner_”;
6 currBannerElem = document.getElementById(prefix+currentBannerID);
7 bannerToChange = document.getElementById(prefix + bannerID);
8 currBannerElem.removeClassName(“active”);
9 bannerToChange.addClassName(“active”);
10 currentBannerID = bannerID;
11 }
12 currentBannerID = 1;
13 changeTimer = setTimeout(changeBanner, 5000);
1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer);
3 changeTimer = setTimeout(changeBanner, 5000);
4
5 prefix = “banner_”;
6 currBannerElem = document.getElementById(prefix+currentBannerID);
7 bannerToChange = document.getElementById(prefix + bannerID);
8 currBannerElem.removeClassName(“active”);
9 bannerToChange.addClassName(“active”);
10 currentBannerID = bannerID;
11 }
12 currentBannerID = 1;
13 changeTimer = setTimeout(changeBanner, 5000);Sequence 1
Sequence 2
1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer);
3 changeTimer = setTimeout(changeBanner, 5000);
4
5 prefix = “banner_”;
6 currBannerElem = document.getElementById(prefix+currentBannerID);
7 bannerToChange = document.getElementById(prefix + bannerID);
8 currBannerElem.removeClassName(“active”);
9 bannerToChange.addClassName(“active”);
10 currentBannerID = bannerID;
11 }
12 currentBannerID = 1;
13 changeTimer = setTimeout(changeBanner, 5000);
1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer);
3 changeTimer = setTimeout(changeBanner, 5000);
4
5 prefix = “banner_”;
6 currBannerElem = document.getElementById(prefix+currentBannerID);
7 bannerToChange = document.getElementById(prefix + bannerID);
8 currBannerElem.removeClassName(“active”);
9 bannerToChange.addClassName(“active”);
10 currentBannerID = bannerID;
11 }
12 currentBannerID = 1;
13 changeTimer = setTimeout(changeBanner, 5000);
JavaScript Fault Localization: Challenges (2) DOM-related JS Errors – fault can be in JS code
or DOM – find direct DOM access
9
direct DOM access
element = $(“elem”);
b = element.getAttribute(“badAttr”)
element.innerHTML = “text”;
b.value = “newValue”;
Scope of Technique Types of DOM-related JS errors
10
Code-terminating DOM-related JS errors
exception
Output DOM-related JS errorsfunction changeToBlue(elem) {
elem.style.color = “red”;
}Wrong colour change
Block Diagram
11
Instrument JS Code
Run Web Application
Generate Traces
Analyze backward slice
Extract Relevant Sequence
Partition into Sequences
Trace file
Web Application
direct DOM access
Trace Collection
Phase
Trace Analysis Phase
Trace Collection
1. Intercept JS code in web app and instrument each line
2. Run web app – instrumentation will execute for each line that executes
3. Generate a dynamic trace of all executed lines
12
- Trace contains line number, function name, and values of in-scope variables
- Error marker also in trace
Instrument JS Code
Run Web Application
Generate Traces
Trace file
Web Application1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer);
3 changeTimer = setTimeout(changeBanner, 5000);
4
5 prefix = “banner_”;
6 currBannerElem = document.getElementById(prefix+currentBannerID);
7 bannerToChange = document.getElementById(prefix + bannerID);
8 currBannerElem.removeClassName(“active”);
9 bannerToChange.addClassName(“active”);
10 currentBannerID = bannerID;
11 }
12 currentBannerID = 1;
13 changeTimer = setTimeout(changeBanner, 5000);
1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer); trace();
3 changeTimer = setTimeout(changeBanner, 5000); trace();
4
5 prefix = “banner_”; trace();
6 currBannerElem = document.getElementById(prefix+currentBannerID);trace();
7 bannerToChange = document.getElementById(prefix + bannerID); trace();
8 currBannerElem.removeClassName(“active”); trace();
9 bannerToChange.addClassName(“active”); trace();
10 currentBannerID = bannerID; trace();
11 }
12 currentBannerID = 1; trace();
13 changeTimer = setTimeout(changeBanner, 5000); trace();
1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer); trace();
3 changeTimer = setTimeout(changeBanner, 5000); trace();
4
5 prefix = “banner_”; trace();
6 currBannerElem = document.getElementById(prefix+currentBannerID);trace();
7 bannerToChange = document.getElementById(prefix + bannerID); trace();
8 currBannerElem.removeClassName(“active”); trace();
9 bannerToChange.addClassName(“active”); trace();
10 currentBannerID = bannerID; trace();
11 }
12 currentBannerID = 1; trace();
13 changeTimer = setTimeout(changeBanner, 5000); trace();
Trace Record Prefix:
changeBanner:::4
Variables:
currentBannerID (global): 1
changeTimer (global): 2
bannerID (local): -11
prefix (local): none
currBannerElem (local): none
bannerToChange (local): none
Trace Analysis
1. Trace is divided into sequences2. Relevant sequence – execution sequence that
contains error marker3. Analyze backward slice of last variable that held
null to see where null value originated. The line where null value originated is the direct DOM access.
13
Instrument JS Code
Run Web Application
Generate Traces
Analyze backward slice
Extract Relevant Sequence
Partition into Sequences
Trace file
direct DOM access
Web Application
1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer);
3 changeTimer = setTimeout(changeBanner, 5000);
4
5 prefix = “banner_”;
6 currBannerElem = document.getElementById(prefix+currentBannerID);
7 bannerToChange = document.getElementById(prefix + bannerID);
8 currBannerElem.removeClassName(“active”);
9 bannerToChange.addClassName(“active”);
10 currentBannerID = bannerID;
11 }
12 currentBannerID = 1;
13 changeTimer = setTimeout(changeBanner, 5000);
Sequences.: (1) line2 -> line3 -> line5 -> line6 -> line7 -> line8 -> line9
(2) line12 -> line13
Trace Analysis: Example1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer);
3 changeTimer = setTimeout(changeBanner, 5000);
4
5 prefix = “banner_”;
6 currBannerElem = document.getElementById(prefix+currentBannerID);
7 bannerToChange = document.getElementById(prefix + bannerID);
8 currBannerElem.removeClassName(“active”);
9 bannerToChange.addClassName(“active”);
10 currentBannerID = bannerID;
11 }
12 currentBannerID = 1;
13 changeTimer = setTimeout(changeBanner, 5000);
14
Relevant Seq.: line12 -> line13 -> line2 -> line3 -> line5 -> line6 -> line7 -> line8 -> line9
1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer);
3 changeTimer = setTimeout(changeBanner, 5000);
4
5 prefix = “banner_”;
6 currBannerElem = document.getElementById(prefix+currentBannerID);
7 bannerToChange = document.getElementById(prefix + bannerID);
8 currBannerElem.removeClassName(“active”);
9 bannerToChange.addClassName(“active”);
10 currentBannerID = bannerID;
11 }
12 currentBannerID = 1;
13 changeTimer = setTimeout(changeBanner, 5000);
1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer);
3 changeTimer = setTimeout(changeBanner, 5000);
4
5 prefix = “banner_”;
6 currBannerElem = document.getElementById(prefix+currentBannerID);
7 bannerToChange = document.getElementById(prefix + bannerID);
8 currBannerElem.removeClassName(“active”);
9 bannerToChange.addClassName(“active”);
10 currentBannerID = bannerID;
11 }
12 currentBannerID = 1;
13 changeTimer = setTimeout(changeBanner, 5000);
bannerToChange being assigned return value of DOM access function
Error Marker
Technique outputs this line as direct DOM access
Last variable to take on null value
Implementation of AutoFLox Trace Collection: Modified versions of existing
tools (InvarScope [Groeneveld et al.] and Crawljax [Mesbah et al.])
Trace Analysis: Written from scratch
15
Evaluation: Research Questions RQ1: How prominent are DOM-related
JavaScript errors in web applications? RQ2: What is the fault localization accuracy of
AutoFLox? RQ3: What is the performance overhead of
AutoFLox?
16
Prominence of DOM-related JS Errors RQ1 Approach: Study bug reports of four web apps
(TUDU, TaskFreak, WordPress, and Google) Only JS-related bugs marked valid or fixed are
studied
17
17%
4%
79%
DOM-related
Non-DOM-relatedUnknown Type
65%
22%
13%
Code-terminatingDOM-related
Output DOM-related
Unknown DOM-related
Research Questions RQ1: How prominent are DOM-related
JavaScript errors in web applications? RQ2: What is the fault localization accuracy of
AutoFLox? RQ3: What is the performance overhead of
AutoFLox?
18
Accuracy of AutoFLox RQ2 Approach: Fault injection experiment on
TUDU, TaskFreak, and WordPress Faults injected: Null exceptions
Mutate the parameters of getElementById and getAttribute methods
19
Accuracy of AutoFLox: Results
WordPress: anonymous functions not supported by AutoFLox at the moment
Direct DOM access in Tumblr also found
20
Web App Total Number of Mutations
Number of direct DOM accesses identified
Percentage identified
TaskFreak 29 29 100%
TUDU 24 24 100%
WordPress 13 7 53.8%
Overall 66 60 90.9%
Research Questions How prominent are DOM-related JavaScript
errors in web applications? What is the fault localization accuracy of
AutoFLox? What is the performance overhead of
AutoFLox?
21
Performance RQ3 Approach: Measure trace collection
overhead in Tumblr; measure time it takes for trace analysis to complete Tumblr used because production code more
complex than development code Results
Trace collection incurred 35% overhead Trace analysis took 0.115 seconds to
complete
22
Conclusion DOM-related JS errors are prevalent Created fault localization tool for JS called
AutoFLox http://ece.ubc.ca/~frolino/projects/autoflox/
Evaluated accuracy and performance of AutoFLox
Future work Relax simplifying assumptions Target Output DOM-related JS errors Automatically fixing DOM-related JS errors
23
Backup Slides
25
Web 2.0 Applications: Document Object Model
- Defines elements contained in webpage
Dependability Problems in JavaScript Big contributing factor: JS interaction with
Document Object Model (DOM)
26
DIV
id: elem
DOM JavaScript Code
var x = document.getElementById(“elem”);var x = document.getElementById(“elme”);
Will return null Inexistent ID
DOM-related JavaScript error
Fault Localization Web application testing
What to do after we find errors? Need to fix them…
Fault localization: Find the root cause of the error
27
x = document.getElementById(“elme”);
changeText(x);
…..
function changeText(domElement) {
y = domElement;
y.innerHTML = ‘New Text’;
}exception
root cause
Running Example
28
1 function changeBanner(bannerID) {
2 clearTimeout(changeTimer);
3 changeTimer = setTimeout(changeBanner, 5000);
4
5 prefix = “banner_”;
6 currBannerElem = document.getElementById(prefix+currentBannerID);
7 bannerToChange = document.getElementById(prefix + bannerID);
8 currBannerElem.removeClassName(“active”);
9 bannerToChange.addClassName(“active”);
10 currentBannerID = bannerID;
11 }
12 currentBannerID = 1;
13 changeTimer = setTimeout(changeBanner, 5000);
Passed with no argument (even though changeBanner needs one argument)
bannerID will be set randomly
Would return null if bannerID out of range
NULL EXCEPTION!
AutoFLox: Scope Types of DOM-related JS errors
Code-terminating DOM-related JavaScript Errors: Errors that occur when DOM access function returns null, undefined, or incorrect value
Output DOM-related JavaScript Errors: Errors that occur when DOM update function sets DOM element’s property to wrong value
29
AutoFLox is concerned with code-terminating DOM-related JavaScript errors
Limitations User has to know how to replicate error User required to specify elements to click
during crawling Multiple JS errors currently not supported
30