making your site easier to use, an in-house usability testing case study

Making Your Website Easier to Use: A Usability Testing Case Study

Jason Samuels, IT director Allison Wickler, web content manager

2

National Council on Family Relations• Established 1938• Publisher of the Journal of

Marriage and Family• 3,400 members

3

NCFR Website - 1938

4

NCFR Website - 2000

• Multicolor HTML tables!• Provided updates on the

organization’s major program areas

• Provided online registration for membership renewal and conference registration

5

NCFR Website - 2006

• Look and feel of the website had evolved

• Lots more information posted about all aspects of the organization

• Still just a brochure website (though a large one)

6

NCFR Website - 2011

• Built on Drupal (Content Management System)

• New features include:– Professional Resource

Library– Degree programs guide– Conference schedule

builder– News feed & event

calendar– Blogs

7

NCFR Website - 2011

The 2011 NCFR website was intended to move our organization’s web presence from being a brochure website to being a go-to place for resources and information about the field that we serve.

8

NCFR Website

• Launched in February 2011 with 3,500 nodes of content– (~ 1,500 of these were faculty listings

for the degree programs guide)

• By December this number had more than doubled

9

April 2012 website survey• What do you like most about the

NCFR website?– quantity and quality of resources

• What do you dislike most about the NCFR website?– trouble finding specific resources– difficult to navigate

• What features or improvements would you like?– clearer organization of the content– greater ease of use

10

Time for some usability testing

11

Optimal Workshop

• Recommended by our web developers

• Offers 3 testing products– OptimalSort (card

sorting)– Treejack (sitemap

testing)– Chalkmark (click

tracking)

• Cost is $109 per month per service (unlimited use)

12

What is sitemap testing?

• Strips out all layout and graphics, only shows your menu choices

• Presents the test taker with a series of questions, asks them where they would navigate to find them

• Simple as that

13

How to conduct a sitemap test

1. Audit your web content2. Produce a draft of the new sitemap3. Setup the testing software4. Invite participants and test5. Analyze results6. Revise sitemap7. Test again8. Analyze, revise, and test one more time9. Implement!

14

Web content audit

• Spreadsheet detailing your site content

• 1st column: numbering system

• 2nd column: title of the page or section

• 3rd column: URL

Template adapted from Content Strategy for the Web, by Kristina Halvorson

15

Draft a new sitemap

• Add a column for the new location

• Can hide URL column if that helps

• Re-arrange items• Pro tip: use a

separate column for each menu level

Template adapted from Content Strategy for the Web, by Kristina Halvorson

16

Setup the testing software

• Can specify when the survey closes (date or maximum number of responses reached)

• Paste in your sitemap tree– If each item is in a different column, the software

will recognize this and structure it accordingly

• Add tasks for users to complete, and choose the “correct” answers for each– Keep it reasonably short, 20 tasks = ~10

minutes

• Customize welcome message, instructions, look, colors and post-test questions

17

Invite participants

• Recruitment - e-news announcements, discussion listservs, and personal appeals brought in ~160 volunteers

• Notification – everyone got a notice when the test opened, and a reminder a couple days before it closed

• Incentive – everyone who completed the test was entered into a drawing for a $50 Amazon gift card. Got ~110 responses per round.

18

Analyzing Results:It’s all about the Pietrees

19

Pietree example # 1

20

Pietree example # 1

21

Pietree example # 1

• Result: 91% Direct Success, 6% Indirect Success, 3% Failure

• Analysis: Erika believed that this question tested successfully, and we agree.

• Testing Action: Remove question from testing round 2.

• Recommendations: Place a "Join NCFR" promotion tile on the "About Us" page, and a "Join Now" button on the membership types page to capture the segment of people who went to those sections looking for this item.

22

Pietree example # 2

23

Pietree example # 2

24

Pietree example # 2

• Result: 4% Direct Success, 8% Indirect Success, 88% Failure

• Analysis: I think most of the answers are legitimately good answers.  I'm not sure why "Family newswire" is the only valid answer listed.

• Testing Action: Re-work this question and test again.

• Recommendations: The intent was to get people to a section where news from the field (not from NCFR is posted). The question was not well-phrased to be clear, but the menu item may also be better off with a clearer title.

25

Big lessons

• Based on the answer patterns we learned some big lessons– Our users look for resources everywhere on our

website, so we did more to prioritize them– Nomenclature is terribly important. Avoid

ambiguous menu titles, and be careful not to name multiple items in a way that seems like they lead to the same thing

– Sometimes there’s just going to be a split. Go with the one that got the bigger share, but make a note to link there from other high-percentage choices

26

Iterate

• Revise the sitemap based on what you learned from the test results

• Replace questions that tested well• Rework questions that tested poorly• Each test completed in a 5-day window• 4-5 day turnaround to the next test• Completed 3 testing rounds in a month

27

Deep breath:Take your time, do it right

28

Take your time, do it right

• In our case the web developers needed time to design and build a revised layout

• Allison worked on the spreadsheet for months, tweaking titles and sitemap structure while using the test results as her guide

29

Ready, set, launch!

30

Launch!

• Because of the extent of our changes, all of the work was done on a staging site

• Two-week content freeze: refrain from updating or double-key in staging

• On launch day our developers copied the staging website over the live site

31

NCFR Website - 2013

• Cleaner design• Fly-out menus assist

navigation• Homepage highlights

enterprise areas• Resources throughout,

but also consolidated in library

32

Questions?