mining the web for design guidelines marti hearst, melody ivory, rashmi sinha uc berkeley
Post on 15-Jan-2016
214 views
TRANSCRIPT
![Page 1: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/1.jpg)
Mining the Web for Design Guidelines
Marti Hearst, Melody Ivory, Rashmi Sinha
UC Berkeley
![Page 2: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/2.jpg)
2
The Usability Gap
196M new Web sites in the next 5 years [Nielsen99]
~20,000 user interface professionals [Nielson99]
![Page 3: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/3.jpg)
3
The Usability Gap
Most sites have inadequate usability [Forrester, Spool, Hurst]
(users can’t find what they want 39-66% of the time)
196M new Web sites in the next 5 years [Nielsen99]
A shortage of user interface professionals [Nielson99]
![Page 4: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/4.jpg)
4
One Solution: Design Guidelines
Example Design Guidelines– Break the text up to facilitate scanning– Don’t clutter the page– Reduce the number of links that must be followed to
find information– Be consistent
![Page 5: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/5.jpg)
5
Problems with Design Guidelines
Guidelines are helpful, but– There are MANY usability guidelines
Survey of 21 web guidelines found little overlap [Ratner et al. 96]
Why?– One idea: because they are not empirically validated
– Sometimes imprecise– Sometimes conflict
![Page 6: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/6.jpg)
Question: How can we identify characteristics of good websites on a large scale?
Question: How can we turn these characteristics into empirically validated guidelines?
![Page 7: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/7.jpg)
•Conduct Usability Studies:Hard to do on a large scale
Find a corpus of websites already identified as good!
Use the WebbyAwards database
![Page 8: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/8.jpg)
• WebbyAwards 2000
• Study I: Qualities of highly rated websites
• Study II: Empirically validated design guidelines
• Putting this into use
Talk Outline
![Page 9: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/9.jpg)
9
Criteria for submission to the WebbyAwards
Anyone who has a current, live website Should be accessible to the general public Should be predominantly in English No limit to the number of entries that each
person can make
![Page 10: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/10.jpg)
10
Site Category
Sites must fit into at least one of 27 categories. For example:
•Arts•Activism•Fashion•Health•News•Radio
•Sports•Music•News •Personal Websites•Travel•Weird
![Page 11: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/11.jpg)
11
Webby Judges
– Internet professionals who work with and on the internet: new media journalists, editors, web developers, and other Internet professionals
– have clearly demonstrable familiarity with the category which they review
![Page 12: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/12.jpg)
12
3 Stage Judging Process
Review Stage: From 3000 to 400 sites– 3 judges rate each site on 6 criteria, and cast a
vote if it will go to the next stage
• Nominating Stage: From 400 to 135 sites– 3 judges rate each site on 6 criteria, and cast a
vote if it will go to the next stage
•Final Stage: From 135 to 27 sites•Judges cast vote for best site
![Page 13: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/13.jpg)
13
Criteria for Judging
6 criteria– Content– Structure & navigation– Visual design– Functionality– Interactivity– Overall experience
Scale: 1-10 (highest) Nearly normally distributed
![Page 14: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/14.jpg)
14
is the information provided on the site.
Good content is engaging, relevant, appropriate for the audience-you can tell it's been developed for the Web because it's clear and concise and it works in the medium …
Content
![Page 15: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/15.jpg)
15
is the appearance of the site.
Good visual design is high quality, appropriate, and relevant for the audience and the message it is supporting …
Visual Design
![Page 16: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/16.jpg)
16
is the way a site allows a user to do something.
Good interactivity is more than sound effects, and a Flash animation. It allows the user to give and receive. Its input/output in searches, chat rooms, ecommerce etc.…
Interactivity
![Page 17: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/17.jpg)
Classification Accuracy for SitesClassification Accuracy for Sites = 91%= 91%
Can votes be predicted by specific criteria?Statistical Technique: Discriminant analysisQuestion: Can we predict the votes from the 5 specific criteria?
Can overall rating be predicted by specific criteria?Statistical Technique: Regression analysisQuestion: What % variance is explained by 5 criteria
Percentage variance explainedPercentage variance explained = 89%= 89%
![Page 18: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/18.jpg)
Review Stage: Which criteria contribute most to overall rating?
Figure 2a. Review StageContribution of Specific Criteria to Overall Site
Rating
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Content Navigation VisualDesign Interactivity Functionality
![Page 19: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/19.jpg)
Nominating Stage Analysis 6 criteria
– Content, Structure & Navigation, Visual Design, Functionality & Interactivity
– Overall experience
400 sites 3 judges rated each site
![Page 20: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/20.jpg)
Nominating Stage: Top sites for each category
Mean = 7.6SD = 1.66
Ove
rall
Rat
ing
overall
10.09.08.07.06.05.04.03.02.01.0
300
200
100
0
![Page 21: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/21.jpg)
Which criteria contribute to overall rating at Nominating Stage?
00.10.20.30.40.50.60.70.80.9
1
Content Navigation VisualDesign Interactivity Functionality
Contribution of Criteria(Correlation)
Unique Contribution of Criteria(Partial Correlation)
77% variance explained in overall rating77% variance explained in overall rating
![Page 22: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/22.jpg)
23
Summary of Study I Findings
The specific ratings do explain overall experience.
The best predictor of overall score is content.
The second best predictor is interactivity.
The worst predictor is visual design
![Page 23: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/23.jpg)
24
Are there differences between categories?
•Arts•Activism•Fashion•Health•News
•Sports•Music•News •Personal Websites•Travel
![Page 24: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/24.jpg)
ArtArts: Contribution of criteria to overall rating
Variance explained = 93%
0.50.550.6
0.650.7
0.750.8
0.850.9
0.951
Content Navigation VisualDesign Interactivity Functionality
![Page 25: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/25.jpg)
Commerce SitesCommerce Sites: Contribution of criteria to overall rating
Variance explained = 87%
0.50.550.6
0.650.7
0.750.8
0.850.9
0.951
Content Navigation VisualDesign Interactivity Functionality
![Page 26: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/26.jpg)
Radio SitesRadio Sites: Contribution of criteria to overall rating
Variance explained = 90%
0.50.550.6
0.650.7
0.750.8
0.850.9
0.951
Content Navigation VisualDesign Interactivity Functionality
![Page 27: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/27.jpg)
28
Conclusions: Study I
The importance of criteria varies by category.
Content is by far the best predictor of overall site experience. Interactivity comes next.
Visual Design does not have as much predictive power except in specific categories
![Page 28: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/28.jpg)
29
Study II
• An empirical bottom-up approach to developing design guidelines
–Challenge: How to go use Webby criteria to inform web page design?
–Answer: Identify quantitative measures that characterize pages
![Page 29: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/29.jpg)
30
Quantitative Measures– Page Composition
• words, links, images, …
– Page Formatting • fonts, lists, colors, …
– Overall Characteristics • information & layout
quality
![Page 30: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/30.jpg)
Quantitative page measures
•Word Count•Body Text %•Emphasized Body Text %•Text Cluster Count•Link Count•Page Size•Graphic %•Color Count •Font Count
![Page 31: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/31.jpg)
32
Quantitative Measures: Word Count
![Page 32: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/32.jpg)
33
Quantitative Measures: Body Text %
![Page 33: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/33.jpg)
34
Quantitative Measures: Emphasized Body Text %
![Page 34: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/34.jpg)
35
Quantitative Measures: Text Positioning Count
![Page 35: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/35.jpg)
36
Quantitative Measures: Text Cluster Count
![Page 36: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/36.jpg)
37
Quantitative Measures: Link Count
![Page 37: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/37.jpg)
38
Quantitative Measures: Page Size (Bytes)
![Page 38: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/38.jpg)
39
Quantitative Measures: Graphic %
![Page 39: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/39.jpg)
40
Quantitative Measures: Graphic Count
![Page 40: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/40.jpg)
Study Design
Low Rated SitesBottom 33%
Webby Ratings
Highly Rated SitesTop 33%
Quantitative Page Metrics
Word Count, Body Text %, Text Cluster Count, Link Count etc.
Model AccuracyAcrossCategories
67%
63%
WithinCategories
83%
76%
![Page 41: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/41.jpg)
Classification Accuracy Comparing Top vs. bottom Accuracy higher for within categories
N Top BottomOverall 1286 67% 63%Community 305 83% 92%Education 368 76% 73%Finance 142 77% 93%Health 165 93% 87%Living 106 42% 76%Services 208 86% 75%Cat. Avg. 76% 83%
Classification Accuracy
Within Categories
![Page 42: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/42.jpg)
Which page metrics predict site quality? All metrics played a role
– However their role differed for various categories of pages (small, medium & large)
Summary– Across all pages in the sample
Good pages had significantly smaller graphics percentage
Good pages had less emphasized body text Good pages had more colors (on text)
![Page 43: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/43.jpg)
44
Role of Metrics for Medium Pages (230 words on average)
Good medium pages – Emphasize less of the body text– Appear to organize text into clusters (e.g., lists and
shaded table areas)– Use colors to distinguish headings from body text
Suggests that these pages– Are easier to scan
![Page 44: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/44.jpg)
45
No Text ClusteringNo SelectiveHighlighting
Low Rated Page
![Page 45: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/45.jpg)
46
SelectiveHighlighting
High Rated Page
Text Clustering
![Page 46: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/46.jpg)
47
Why does this approach work?
Superficial page metrics reflect deeper aspects of information architecture, interactivity etc.
![Page 47: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/47.jpg)
48
Possible Uses
A “grammar checker” to assess guideline conformance
Imperfect Only suggestions – not dogma
Automatic template suggestions Automatic comparison to highly usable
pages/sites
![Page 48: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/48.jpg)
49
Current Design Analysis Tools
Some tools report on easy-to-measure attributes
– Compare measures to thresholds
– Guideline conformance
![Page 49: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/49.jpg)
50
Comparing a Design to Validated Good Designs
Web Site Design
•Prediction•Similarities•Differences•Suggestions
Analysis Tool
Profiles
Comparable Designs
Favorite Designs
![Page 50: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/50.jpg)
51
Future work
Distinguish according to page role– Home page vs. content vs. index …
Better metrics– More aspects of info, navigation, and graphic design
Site level as well as page level Category-based profiles
– Use clustering to create profiles of good and poor sites – These can be used to suggest alternative designs
![Page 51: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/51.jpg)
52
Conclusions: Study II
Automated tools should help close the Web Usability Gap
We have a foundation for a new methodology– Empirical, bottom up
We can empirically distinguish good pages– Empirical validation of design guidelines– Can build profiles of good vs. poor sites
Eventually build tools to help users assess designs
![Page 52: Mining the Web for Design Guidelines Marti Hearst, Melody Ivory, Rashmi Sinha UC Berkeley](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649d7f5503460f94a6378a/html5/thumbnails/52.jpg)
53
More Information
http://webtango.berkeley.edu