Al l i so n Co w g i l l & Aman d a D i n sco re

Hen ry Mad d en L i b ra ry

Ca l i fo rn i a S ta te Un i vers i t y, F resn o

C a l i f o r n i a A s s o c i a t i o n o f R e s e a r c h L i b r a r i e s

A p r i l , 2 0 1 0

S a c r a m e n t o , C A

Usability Testing of a Library Web Site: Librarians and Anthropologists Working Together

University of Rochester Study

Ethnographic approach to studying campus libraries and their users

The Library Study at Fresno State

September, 2008 - May, 2009

Goal: “to discover information about student life that campus librarians could use to both increase library usage and improve student users’ experience of library services.”

“Draw How You Feel When You Write a Paper.”

Library Study Recommendations

Provide different search boxes for different kinds of holdings. Attempt a redesign that aims for a less institutional, more youthful and

content-rich feel. Move toward a Web 2.0 model.

Still Needed to know…

How do students use the site? What tasks can students accomplish? What assumptions are we making? What can we discover that will inform our re-design?

Web Usability Workgroup (WUW)

Decision to conduct a usability study and the formation of the Web Usability Workgroup (WUW)

Involvement of Anthropology professor, Hank Delcore, and his students in ANTH 111B (Intermediate Ethnographic Fieldwork)

“How people develop and use technological solutions (like the library's website)

is one of the most complicated and important areas of modern life, so it's a

natural anthropological attention.” Delcore, 2010

Usability Testing

Usability Testing demonstrates how users interact with the system.

Participants are asked to complete a series of pre-defined tasks and to “think aloud” while completing them.

Benefits: See what real users actually do (rather than what they say they do) Provides quantitative data rather than opinion or conjecture Participants complete common tasks (finding a book, article, etc…)

Process

Develop a plan

Purpose Determine the user group to be tested Pre-survey/Consent form Tasks to be performed (number & time) Script & Moderator Instructions—ensures consistency Test Design—how it will be conducted and how results are to be

recorded

Pre-test Recruit & test Analyze the results

Subjects: Juniors

Sample Size: 42 (plus 10 pre-tests)

Recruitment: On-site

People: 5 student moderators, 5 WUW members (Anthropology and library faculty)

Facilities: 4 small rooms for testing; welcome area

Equipment: 4 Macs

Software: Screenium

Test Documents: Pre-Survey/Consent Form Introduction Script Moderator Instructions Usability Test Activity

Cost: $636 ($520 for Starbucks gift cards; Screenium software site licenses $29 per computer)

Nuts & Bolts

Benefitted from pre-testing and multiple run-throughs with team members and moderators

Challenges with script, wording of questions, issues that we hadn’t previously considered (e.g. using Google)

Benefits of using student moderators (objective; offered valuable insight)

Talk aloud method was sometimes challenging

Testing took less time than we thought

On-site recruitment worked better than expected

The $10 Starbucks card incentive worked well

Subjects took it seriously Software (Screenium) worked

well

Reflections

Coding & Data Entry

Subject Number, time stamp, task number (1-10)

Link Label (words clicked) & URL

Notes (including quotes from subjects)

“Lostness” Codes for task completion Codes for positive & negative

comments

Codes for non-verbal behaviors such as smiling, frowning, laughing

Data Analysis (in process)

Discussion