Automated Evaluation of

WordPress Theme Design

Derek Ohanesian

CSC 499Advisor: Prof. Chris Fernandes

November 7, 2014

Research QuestionIn what ways can an automated system provide web design advice?

Can an automated system analyze a blog’s WordPress theme and provide valuable feedback about design improvements?

Motivation

The World Wide Web democratizes publishing

A well designed website is less accessible.

What is design?

By design, we mean the user interface, encompassing the presentation, aesthetics, and accessibility of the website.

Design is separate from the content of the website.

Why Blogs?• Limits scope

• Similar content, differentiated by design.

• Focus on evaluating the design, while the content type remains constant

• Limited data set the two-column style blogs only

What is WordPress?• An open-source content management system

(CMS) for blogging. The most popular CMS for websites.

• Separates the content from the design. Different WordPress “themes” can be applied to the blog content to create the design of the website.

• Thousands of freely available and open-source themes available.

Research Question

How can we use data collected from WordPress themes to provide valuable feedback about design improvements?

Data Collection• WordPress themes include a set of functions that

the theme designer can call to access the blog’s content and present it in the design of the page

• I intercept these function calls and inject labels to each section of the website, so I know what content is being displayed

Title

Tagline

Post Title P

ost

Post titles

Data Collection• By injecting labels into the WordPress theme, I

was able to collect more precise data about the content being displayed.

• In addition, general data about…o position of elementso fontso colorso sizeso visual style

Procedure

• Previous research shows that certain quantitative heuristics correlate with good design

• For this project, I tested similar heuristics on a data set of over 200 two-column WordPress themes

Heuristics• Quantity of unique colors used

• Text contrast on background

• Left/right balance of major elements

• Density of content elements (percent of area)

• Font size for some elements

• Quantity of fonts used

Number of Colors

0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930313233343536373839400

5

10

15

20

25

30

35

40

45

50

Number of Colors Used

NumberOf Themes

Number of Colors

0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930313233343536373839400

5

10

15

20

25

30

35

40

45

50

Number of Colors Used

NumberOf Themes

One theme with 38 colors

ColorScheme

5 colors

17 Colors

Number of Colors

• “Sweet spot” at 5 colors?

• Greater than 10-15 colors indicative of a problem?

Layout Heuristics

• Area of page used (density)

• Balance (left/right)

Area of Page (~2% content)

Area of Page (~47%)

Area of Page (~47%)

Balance

Balance

100%

(of content on left side)

Balance

50%(of content on left side)

Text Contrast• WC3 recommendation for making web more

accessible

• Web Content Accessibility Guidelines (WCAG)

• Recommended color contrast algorithm must produce contrast within a certain range to be considered “good visibility”

Text Contrast

100% of color combinations pass WCAG test

Text Contrast

0% of color combinations pass WCAG test

Text Contrast

• Inconsistent data

• Some color combinations are not included in my analysis (hard to parse CSS, or alpha channels)

• WCAG 2.0 has an improved color-contrasted algorithm

Evaluating Heuristics

• Heuristic provide a set of scores for each theme

• A test set of unevaluated themes has been set aside

• A human evaluation of the themes in the test set can be compared to the automated evaluation to evaluate its effectiveness

Future work• Evaluating automated scores against human

heuristic evaluation (this term)

• Assigning weights to heuristics

• Scoring themes with additional heuristics

• Adding additional themes to data set

Q & A