Web Site Usability Study

John GottfriedSpring 2008

By way of illustration…

And there are, well,

other Web sites…

There are really good Web sites…

Web Site Evaluation

• Compatibility Testing– Interaction among components (browsers,

computers, operating systems, printers, etc.)

• Security Testing– Checks for unauthorized access to system

• Functional Testing– Looks for broken features or functions

• Usability Testing– Does the user find navigating the site difficult,

unpleasant or inefficient?

Kaner, C., & Fiedler, R. (2005). Testing library Web sites for usability. Knowledge Quest, 33(3), 29-31.

Web Site Evaluation…Why Not

• Time consuming

• Can be expensive

• Intimidating process for the uninitiated

• What are you measuring?

• Inconclusive (results, but no solutions)

Benjes, C., & Brown, Janis F. (2001). Test, revise, retest: Usability testing and library Web sites . Internet References Quarterly, 5(4), 37-54.

Web Site Evaluation…Why• Overcome subjective values & politics

• Stakeholder input and feedback

• Practical decision-making

• Reduce cost of development & support

• Justify investment

• Marketing & public relations

--Benjes, C., & Brown, Janis F. (2001). Test, revise, retest: Usability testing and library Web sites . Internet References Quarterly, 5(4), 37-54.--Dicks, R. S. (2002). Proceedings of the 20th Annual International Conference on Computer Documentation, Toronto, Ontario, 26-30.--Travis, D. (2001, October 10). A business case for usability. Retrieved 12/4/2007 from: http://websitetips.com/articles/usability/benefits/

Web Site Evaluation…How

Test Method # of Libraries % of Libraries

Cognitive Walkthrough 27 52%

Individual Staff Review 26 50%

Focus Groups 23 44%

Online or Web Survey 21 40%

Usability Testing 19 37%

Web Server Log Analysis 19 37%

Popp, M. P. (2001, March 15-19). Testing library Web sites: ARL libraries weigh in. Paper presented at ACRL Tenth National Conference, Denver, CO.

Study background

• Indiana University Libraries, Indianapolis, IN• Urban Campus, 29,000 students, IU Medical School

• Purpose—evaluate effectiveness of new Library Web site

• Recruited 30 Volunteers ($10 Campus Cards)• Method (Battleson, Booth & Weintrop)

• Basic information about Ss. Role, computer and library use• Completion of 10 common Library Web Site tasks

– Recorded time, task completion, comments and # of pages

• Ss. comments and reactions to Web Site

Battleson, B., Booth, A., & Weintrop, J. (2001). Usability testing of an academic library Web site: A case study. Journal of Academic Librarianship, 27(3), 188-198.

Participants

• 2/3 of participants were undergraduates

• Only 1 student in first semester, 2/3 over one year

• All but one visit the Library at least once per week & over half use the Library Web site at least once per week

• All but one were confident computer users, and all but one use the Internet daily

• Heavy users, not novices

Student Comments

• Not many specific comments• Overall, the Web site was received enthusiastically (25)

o Helpful, useful (6)• Biggest negative was finding articles (11)

o Several suggestions for a Help button for articles• Biggest positive was Ask a Librarian (11)

o Many users were not aware of this pageo Unaware users said they would use it now

• International students: need instruction sessions• Instruction:

• Challenging first-year class/instructor a key factor for extraordinary overall performance

• In-class librarian instruction a key factor task proficiency

Key Implications for Teaching, Learning and Research

• Students need considerable help finding articles and periodicals

• Placement and awareness of subject guides• Fast, easy access to research help

• Instruction is a critical component of successful performance• In-class librarian instruction for task proficiency• Curriculum-based for overall effectiveness

• International users may need specialized instruction in basic terms and concepts 

Key Implications for Web Site Design

• Eliminate the use of confusing terms and jargon– Check user comprehension of terms– Recruit diverse sample of students, including

international students– Use icons and images where possible

• Key functions must be clearly marked and labeled, and easily accessible

• Students are currently required to “drill down” too far to find articles

General Recommendations• User focus

– Research, customization, clear language

• Web design– Easy-to-use, well-maintained, limit complexity

• Value added– Assistance, annotations, tutorials, subject guides

• Links to free materials– e.g., other libraries, open Web content

Rich, L. A., & Rabine, J. L. (1999). How libraries are providing access to electronic serials: A survey of Academic library Web sites. Serials Review, 25(2), 35-46.

Wright, C. A. (2004). The academic library as a gateway to the Internet: An analysis of the extent and nature of search engine access from the academic library home pages. College & Research Libraries, 65(4), 278-286.

Detlor, B., & Lewis, V. (2006). Academic library Web sites: Current practice and future directions. The Journal of Academic Librarianship, 32(3), 251-258.

Liu, Shu. (2008). Engaging users: The future of academic library Web sites. College & Research Libraries, 69(1), 6-27.

QUESTIONS?

Contact:

John Gottfried Business Librarian SUNY Old Westbury

Voice: 516-876-2895 Email: gottfriedj@oldwestbury.edu