x-pert: accurate identification of cross-browser issues in web applications
DESCRIPTION
X-PERT: Accurate Identification of Cross-Browser Issues in Web Applications. Shauvik Roy Choudhary, Mukul Prasad, Alessandro Orso. Labs of America. Partially supported by. Web Applications. Mozilla Firefox. Internet Explorer. Mozilla Firefox. Internet Explorer. Google Chrome. - PowerPoint PPT PresentationTRANSCRIPT
X-PERT: Accurate Identification
of Cross-Browser Issues in Web Applications
Shauvik Roy Choudhary, Mukul Prasad, Alessandro Orso
Labs of America
Partially supported by
Web Applications
2
3
4
Mozilla Firefox
Internet Explorer
5
Mozilla Firefox
Internet Explorer
6
Google Chrome
7
Apple Safari
8
Apple Safari Mozilla Firefox
9
Mozilla Firefox
10
Mozilla Firefox
Internet Explorer
11
Mozilla Firefox
Internet Explorer
?
?
Web Developer
Web Developer
Web Developer
1211Browser versions last
year
Source: StatCounter.com
56Browser versions last
yearwith at least 1% market share
150mn visits/month (StatCounter.com)
Challenges
Manual inspectionis expensive
DOM differs between browsers
Mimic end user’sperception
Produce readable, effective & actionable
reports
EngineeringIssues…
Modern apps havemany dynamic screens
Detection ofCross-Browser Issues
(XBI)
Web Application
Model Generation
Model Comparison
Error Report
δ
δ
Model Definition and Comparison
WebDiff [ICSM’10]Roy Choudhary and Orso
CrossCheck [ICST’12]Roy Choudhary, Prasad and Orso
CrossT [ICSE’11]Mesbah and Prasad
Screen Transition Graph
Screen Model
Screen image + geometry
DOM Tree
Study of Real World Cross-Browser Issues
(XBIs)• Examined 100 websites• Found 23 with XBIs
• XBI TypesoBehavioral XBIs
oContent XBIs
oStructural XBIs
9% 22
%30%
57%
TextVisual
Random URL Generator
Limitations of existing techniques
• Miss Structural XBIs
• Focus on symptoms rather than causeso Duplicate reportso Low Precision
ComprehensiveXBI Detection
• Behavioral XBIs
• Content XBIs
• Structural XBIs
TextVisual
Example
Example
Example
Behavior XBI9%
ExampleContent
XBI(Visual) –
30%
Content XBI
(Text) – 22%
Structural XBI57%
ComprehensiveXBI Detection
• Behavioral XBIs
• Content XBIs
• Structural XBIs
TextVisual
ComprehensiveXBI Detection
• Behavioral XBIs
• Content XBIs
• Structural XBIs
TextVisual
Behavior check using Graph Isomorphism
= ?CrossCheck [ICST’12]Roy Choudhary, Prasad and Orso
CrossT [ICSE’11]Mesbah and Prasad
ComprehensiveXBI Detection
• Behavioral XBIs
• Content XBIs
• Structural XBIs
TextVisual
Text Contentcheck using String equality
= ?
WebDiff [ICSM’10]Roy Choudhary and Orso
CrossCheck [ICST’12]Roy Choudhary, Prasad and Orso
CrossT [ICSE’11]Mesbah and Prasad
ComprehensiveXBI Detection
• Behavioral XBIs
• Content XBIs
• Structural XBIs
TextVisual
Visual Contentcheck using Image Matching
= ?Computer
Vision
WebDiff [ICSM’10]Roy Choudhary and Orso
CrossCheck [ICST’12]Roy Choudhary, Prasad and Orso
Detecting Structural XBIs
• Intuition: Disruption in the relative arrangement of web page elements leads to Structural XBIs
• Idea: Abstraction to represent relative arrangement and compare it to expose errors
Alignment Graph (AG)
• A graph capturing structural relationships between elements (rectangles) on the webpage
• An AG has:oContainment relationshipsoAlignment relationships oAttributes
aboveright-alignleft-align
Step 1: AG Construction
(x1, y1) (x2,
y2)
1
2
3
4
56
7
8
9
10
11
12
13
14
15
16
17
18
Step 1: AG Constructiona. Containment Relationship
e contains e’ iff• ( x1 ≤ x1’ ∧ y1 ≤ y1’ ∧ x2 ≥ x2’ ∧ y2 ≥ y2’ ) , and• if same bounds and XPath(e) ≤ XPath(e’)
(x1, y1)
(x2, y2)
(x1’, y1’)
(x2’, y2’)
e
e’
Step 1: AG Constructiona. Containment Relationship
Step 1: AG Constructionb. Alignment Relationship
Step 1: AG Constructionc. Attributes
Button 1 Button 2
What alignment attributes can we infer?
• Button 1 is on the left of Button 2
• Button 1 and Button 2 have their tops aligned
• Button 1 and Button 2 have their bottoms aligned
leftOftop-alignbottom-align
y1 = y1’
y2 = y2’
x2 < x1’
Step 1: AG Constructionc. Attributes
Button 1 Button 2
leftOftop-alignbottom-align
Step 2: Matching AGs
Step 2: Matching AGs
TextVisual
ComprehensiveXBI Detection
• Behavioral XBIs
• Content XBIs
• Structural XBIs
TextVisual
ReportGeneratio
n
Model Generatio
n
BehaviorChecker
ElementMatcher
Structure
Checker
ContentChecker
Model Comparison
✔ ✔
✔
Empirical Evaluation
Tool: X-PERT (Cross-Platform Error ReporTer)
Research Questions:• RQ1: Can X-PERT find XBIs in real web
applications?
• RQ2: How does X-PERT compare to the state-of-art?
Experimental Protocol
v14.0.1 v9.0.9v/s
Ran X-PERT on Subject Applications
Subjects
+ = XBIs(Ground Truth)Manual Check
Error Reports Manual Check
=False Positives & Negatives+
Subjects
Name Type States TransitionsDOM Nodes (per screen)
max min average Organizer Productivity 13 99 27482 10001 13051 GrantaBooks Publisher 9 8 37800 15625 25852 DesignTrust Business 10 20 26437 7772 18694 DivineLife Spiritual 10 9 140611 9082 49886 SaiBaba Religious 13 20 42606 524 12162 Breakaway Sport 19 18 45148 8191 13059 Conference Information 3 12 878 817 853 Fisherman Restaurant 15 17 39146 15720 21336 Valleyforge Lodge 4 12 5416 4733 5046 UniMelb University 9 8 15142 12131 13792 Konqueror Software 5 4 17586 15468 16187 UBC Club 7 7 20610 7834 12094 BMVBS Ministry 5 20 19490 12544 15695 StarWars Movie 10 9 28452 19719 22626
800 - 140K elements
Prior Art (6)
Example and Survey (4)
Random (4)
Effectiveness
NAME XBIBEHAVIOR STRUCTURE
CONTENTTOTALTEXT IMAGE
TP FP TP FP TP FP TP FP TP FPOrganizer 10 1 0 9 0 0 0 0 0 10 0
GrantaBooks 27 16 0 11 0 0 0 0 0 27 0
DesignTrust 7 2 0 5 3 0 0 0 0 7 3DivineLife 11 7 0 3 6 1 0 0 0 11 6SaiBaba 5 2 0 2 9 0 0 0 0 4 9Breakaway 13 0 0 10 2 0 0 0 0 10 2Conference 7 2 0 3 0 1 0 1 0 7 0Fisherman 5 1 0 3 1 0 1 1 0 5 2Valleyforge 3 0 0 2 2 0 0 1 0 3 2UniMelb 2 2 0 0 0 0 0 0 1 2 1Konqueror 0 0 0 0 0 0 0 0 6 0 6UBC 0 0 0 0 0 0 0 0 0 0 0BMVBS 1 0 0 0 0 0 0 0 0 0 0StarWars 12 0 0 12 0 0 0 0 0 12 0TOTAL 103 33 0 60 23 2 1 3 7 98 31
RQ1: Can X-PERT find XBIs in real web applications?
Answer: Yes
Improvement
NAME XBIX-PERT CROSSCHECK
Prec FP Recl Dp Prec FP Recl DpOrganizer 10 100% 0 100% 0 80% 2 80% 13GrantaBooks 27 100% 0 100% 0 96% 1 100% 0DesignTrust 7 70% 3 100% 0 5% 122 86% 3DivineLife 11 65% 6 100% 0 29% 24 91% 3SaiBaba 5 31% 9 80% 0 7% 53 80% 10Breakaway 13 83% 2 77% 1 13% 49 54% 12Conference 7 100% 0 100% 0 100% 0 100% 0Fisherman 5 71% 2 100% 0 44% 5 80% 8Valleyforge 3 60% 2 100% 0 50% 1 33% 0UniMelb 2 67% 1 100% 0 7% 27 100% 0Konqueror 0 NA 6 NA 0 0% 11 NA 0UBC 0 NA 0 NA 0 0% 1 NA 0BMVBS 1 NA 0 NA 0 0% 2 0% 0StarWars 12 100% 0 100% 0 10% 91 83% 3
TOTAL 103 77%31 95% 1 32% 389 74% 52
X-PERT
Precision = 77% (45%)
Recall = 95% (14%)
RQ2: How does X-PERT compare to the state-of-art?
Answer: X-PERT has better precision and recall
Experimental Data & Tool
• Releaseo Experiments (Crawl Data + Reports)o Layout testing algorithm
implementation
http://gatech.github.io/xpert
51
Related Worko Industrial tools
• Test Suites for Web Browsers – Acid and test262
• MS Expression Web, BrowserShots, BrowserStack
• Browsera, MogoTest, BrowserBite
o Research techniques
• Eaton & Memon [IJWET’07]
• Dallmeier, Burger, Orth & Zeller [JSTools’12]
Contributions
Empirical evaluationshows effectiveness
New Structural XBIDetection Algorithm
Comprehensive Technique to detect XBIs
Study of Real World XBIs
Thank YouShauvik Roy Choudhary, Mukul Prasad, Alex Orso