assessing the effort of repairing the accessibility of web sites
DESCRIPTION
TRANSCRIPT
![Page 1: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/1.jpg)
ASSESSING THE EFFORT OF REPAIRING THE ACCESSIBILITY OF WEB SITES
Nádia Fernandes, Luís CarriçoUniversity of Lisbon
ICCHP, Linz, Austria, July 11-13, 2012
![Page 2: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/2.jpg)
ICCHP, Linz, Austria, July 13, 2012 2
![Page 3: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/3.jpg)
ICCHP, Linz, Austria, July 13, 2012 3
Introduction• 40-50% of the Web content uses templates.
• Evaluations are performed in pages as a whole.
• Obfuscating results.
• Available metrics may be misleading.
![Page 4: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/4.jpg)
ICCHP, Linz, Austria, July 13, 2012 4
Objectives1. Assess the effort of repairing a site’s accessibility that was
originally developed using templates
Requirements:
• Develop a new metric, • Template detection algorithm
2. Introducing accessible templates as a form of rapidly repairing a page
![Page 5: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/5.jpg)
ICCHP, Linz, Austria, July 13, 2012 5
The Accessibility Repairing Effort Metric (AREM)
“Primitive elements”- elements of a page, when an element that is part of a template is only considered once.
The metric considers the sum of the number of fails and warnings reported by accessibility evaluation techniques, excluding repeated instances.
![Page 6: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/6.jpg)
ICCHP, Linz, Austria, July 13, 2012 6
The Accessibility Repairing Effort Metric (AREM)
The purpose of this metric is:
1. assess the quality measurement of the accessibility of site construction (effort person.month);
2. and not the perceived quality of the site towards end-users.
![Page 7: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/7.jpg)
ICCHP, Linz, Austria, July 13, 2012 7
Basis:
• QualWeb evaluator
• Fast Match algorithm
The Platform for Accessibility Evaluation
![Page 8: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/8.jpg)
ICCHP, Linz, Austria, July 13, 2012 8
Procedure1. The DOM trees are obtained
2. The pages are compared using the Fast Match algorithm • Comparative function • Result: The “primitive” nodes
3. The accessibility evaluation is executed (QualWeb evaluator)
4. Reports generation (new reports)
5. The repairing effort estimation according to AREM is computed.
![Page 9: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/9.jpg)
Experimental Study• Objective: understand the advantage of using AREM and
validate the template detection method.
• Object of study: 15 sites with templates (Alexa Top 100)
ICCHP, Linz, Austria, July 13, 2012 9
5 low level
5 medium level
5 high level
![Page 10: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/10.jpg)
ICCHP, Linz, Austria, July 13, 2012 10
Experimental Study• QualWeb was applied with the template aware option set.
• The values for the AREM using the element primitiveness.
• We used a conservative rate metric to obtain values of accessibility quality to compare with AREM.
rate conservative =
![Page 11: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/11.jpg)
ICCHP, Linz, Austria, July 13, 2012 11
Metrics Results – AREM (I)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 150
100002000030000400005000060000700008000090000
100000
Non-primitive PrimitiveWeb sites
1 – Higher level of template usage, 15 – Lower level of template usage
![Page 12: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/12.jpg)
ICCHP, Linz, Austria, July 13, 2012 12
Metrics Results - Conservative rate (I)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 150.000.050.100.150.200.250.300.350.40
Non-primitive Primitive
Web sites
1 – Higher level of template usage, 15 – Lower level of template usage
![Page 13: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/13.jpg)
ICCHP, Linz, Austria, July 13, 2012 13
Metrics Results – AREM (II)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 150
100002000030000400005000060000700008000090000
100000
Non-primitive PrimitiveWeb sites
1 – Higher level of template usage, 15 – Lower level of template usage
![Page 14: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/14.jpg)
ICCHP, Linz, Austria, July 13, 2012 14
Metrics Results - Conservative rate (II)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 150.000.050.100.150.200.250.300.350.40
Non-primitive Primitive
Web sites
1 – Higher level of template usage, 15 – Lower level of template usage
![Page 15: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/15.jpg)
ICCHP, Linz, Austria, July 13, 2012 15
Metrics Results - Conservative rate (III)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 150.000.050.100.150.200.250.300.350.40
Non-primitive Primitive
Web sites
1 – Higher level of template usage, 15 – Lower level of template usage
![Page 16: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/16.jpg)
ICCHP, Linz, Austria, July 13, 2012 16
Validating the Approach
Web sites % template % errors on template detection
Higher level of template detection
56% 14%51% 8%
Mid level of template detection
33% 6%31% 12%
Lower level of template detection
18% 7%15% 1%
The number of incorrect elements detected by Fast Match algorithm is less than 10% (average).
![Page 17: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/17.jpg)
ICCHP, Linz, Austria, July 13, 2012 17
Discussion• The validation of the template detection yielded a deviation.
• The algorithm had more failures in sites which use more template based components.
• Regarding the AREM metric:• The difference between the computed values is high; • Less 30% of repairing issues (primitive elements);• Depending on the sites this value can decrease substantially.
• These results confirm and support our previous experiment’s results.
![Page 18: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/18.jpg)
ICCHP, Linz, Austria, July 13, 2012 18
Conclusion• Templates can be very important, reducing the effort of
correction.
• The metric defined is a real indicator of the work that have to be done, unlike certain quality metric that can be misleading.
• We performed a validation experiment of both metric and framework and conclude that the template detection algorithm has a high efficacy.
![Page 19: Assessing the Effort of Repairing the Accessibility of Web Sites](https://reader035.vdocuments.us/reader035/viewer/2022070304/54c49e724a795972468b4623/html5/thumbnails/19.jpg)
ICCHP, Linz, Austria, July 13, 2012 19
Future Work1. Improvements of Fast-Match algorithm to guarantee a
higher accuracy level;
2. A large-scale evaluation of the fast match algorithm.