library website usability: how a library’s site content is...

Library Website Usability: How a Library’s Site Content is Organized

Makes a Difference

Janette Dauenhauer INFO 522: Information Access & Resources

March 16, 2012

Janette Dauenhauer 2

Introduction and Scope

Libraries have been doing usability studies on library Web sites at least

since 1995. For the most part, early studies appearing in the library literature

focused on what a library site should look like, or its interface, with the primary

focus being the library home page. Around this time, Web developers began to

apply the concept of information architecture from systems design to Web site

design. By 1999, libraries began to take notice, embracing the notion that

navigability and content organization was as important as appearance to

usability. Following are articles that discuss the birth of these concepts in

libraries, how they have evolved over the last 12 years, and where the field may

be going in the future.

Care was taken to select articles that directly address the concepts of

content organization and navigability of a library Web site as they pertain to

usability. The articles chosen show the development of these concepts into a

discipline with research methods, analysis, and measurable outcomes. It can be

demonstrated that the success of a library can be directly linked to the success of

its Web site.

Description

Usability testing as it pertains to Web Sites can entail many exercises.

Often, it refers to task-based questions posed to representative users of a site to

determine if the users can achieve a defined goal. It can also involve interviews,

surveys, and card sorting. Chen, Germain, & Rorissa (2011) offered a formal

definition of usability testing as follows:

Usability means that a system has visible working

functionality familiar to its users, maximum reliability, and useful

content that is supported by its environment and aligned with

context of use. In addition, a usable system accommodates the

cognitive capacity and various needs of its users, so that they can


easily understand, effortlessly learn, and dynamically

interact with the system as well as its content, resulting in a

satisfactory experience with a high level of productivity (p. 621).

Implied in this definition is the role of the user and how hard the user must work

to use a given system, or, as discussed this collection of research, a library Web

site. Whether a site is “easily understood” and effortless to use is intrinsically

linked to the site’s information architecture. Traditional information architecture

looks at making a plan, or “blueprint,” to serve as a guide for future

development; labeling; and content organization. Studies of the role of

information architecture and how it can be measured by usability testing have

emerged in library literature over the last 12 years.

Summary of Findings

The concept of usability arrived in the field of Web site development

almost as soon as there was a World Wide Web. In general, usability looked at

whether a user could find what he/she was looking for on a Web site. Many

times, Web designers focused usability testing on the appearance of their sites: If

the site looked good—that is, if content were well-placed on the page, and the

site had a uniform look and feel—it must work well and the user will be

satisfied, they assumed. Indeed, many an award-winning Web site had been

deemed successful by dint of having won an award (Gullikson, Blades, Bragdon,

McKibbon, Sparling, & Toms, 1999, p. 293).

Information architecture as it pertained to systems design had been

around for decades, and it didn’t take long before systems designers began to

note that the Web was a representation of a system of information. Once the idea

of information architecture became married to the Web, library Web developers

took notice.

And why shouldn’t librarians notice this? Librarians are natural-born

information architects, with their skills at categorizing, organizing, searching and


using classification schema and taxonomies (Veldof & Nakerud, 2001, p. 18). It is

no surprise, then, that one of the first studies to look at usability in terms of

information architecture was by librarians, Gullikson, Blades, Bragdon,

McKibbon, Sparling, & Toms (1999). The authors were among the first in the

scholarly literature to note that information architecture influences whether users

will find the information they are looking for, and that their success influences

their decision to come back. This concept of user satisfaction leading to repeat

visits and its integral place in the definition of usability has also been explored

and measured by Jeng (2005). According to Jeng (2005) there was a direct

correlation between effectiveness and the number of clicks a user made and a

strong correlation between effectiveness and satisfaction (p. 111). Robins and

Kelsey (2002) examined error rate, number of clicks, and amount of time to

achieve goals as measures of user satisfaction. Dee and Allen (2006) used a

combination of surveys and proxy server analysis to determine perceived

satisfaction and actual success as measured by number of clicks and time spent

accomplishing a task. Cognizant that many layers of hierarchy mean many

clicks to retrieve information, Swanson (2001) based his decision to create a

shallow hierarchy on the notion of “depth and breadth,” with many links at the

top (home page) and few layers (p. 6).

The challenge librarians face today is a world that is growing ever more

complex, having to provide an increasing number of disparate electronic

resources and information to an audience suffering from information overload

(King & Jannick, 2005, p. 235). The emerging theme from the literature is that

librarian Web site designers must include the user in the design, so that use of

library Web sites is, as Chen, Germain, & Rorissa (2011, p. 621) noted above,

reliable and effortless. Task-based usability studies can validate whether a site is

usable (McGillis, 2001), but how does one develop a navigable site where the

user finds what he/she is looking for? Gullikson, Blades, Bragdon, McKibbon,

Sparling, & Toms (1999), Dickstein and Mills (2000), and Maloney and Bracke

(2004) support the idea of creating a “blueprint” to serve as a guide during the


development process. This guide serves to define user and functional

requirements, network limitations, label conventions, and navigational structure

(e.g., search tools) (Maloney & Bracke, 2004, p. 147). Above all, the user is at the

center of the design. Dickstein and Mills in particular noted that, as part of a

user-centered approach, librarians must resist the urge to think like librarians

and involve their users in the process. To begin the process, librarians should

conduct user needs assessments with stakeholders to inform the scope of the

content and requirements (Duncan & Holliday, 2008, p. 303).

Though librarians may be good at classifying and categorizing, they may

have a tendency to use library “jargon,” often misunderstood or meaningless to

users (Maloney & Bracke, 2004, p. 147). Another common pitfall is the

temptation to organize a library Web site the same way the physical library space

is organized (e.g., by department). In doing so, librarians overlook the reason a

user has come to the site—to look for information, not to look for a tool (McGillis

& Toms, 2000, p. 365). Librarians can engage their users in providing labels and

structure that makes sense to them through card sorting exercises. Dickstein and

Mills (2000) were among the first in the literature to document using this

technique with members of their target audience, but they were soon followed by

many others. How to analyze the results included the informal “eyeballing”

used by Dickstein and Mills, as well as spreadsheet analysis (Turnbow,

Kasianovitz, Snyder, Gilbert, & Yamamoto, 2005) and statistical analysis. Whang

(2008) discusses the various methods of analysis and appropriateness of each as

determined by set size. Robbins, Esposito, Kretz, & Aloi (2007) attempted to deal

with a wide range of results by grouping according to synonyms, and calculating

correlations. Lewis and Hepburn (2009) reported success using factor analysis.

Most studies validated the results of the card sort using task-based usability

testing, but Whang (2008) performed the card sort after usability testing revealed

problems with his site’s labels and structure. Nevertheless, he validated the

results of his card sort results with a follow-up survey.


Current trends in research see the focus drifting toward Google, and the

impact it has on the way people use Web sites. The current wisdom says that

users have been “Googlized” (Swanson & Green, 2011), bypassing a site’s

navigation altogether in favor of prominently placed search boxes on site home

pages. McHale asked whether users would even bother with a library home

page and go straight to Google or Yahoo (2007, p. 142). Augustine and Greene

wondered as early as 2002 whether students weaned on Google would prefer the

simplicity and familiarity of Google search or take a chance of the unknown of a

site’s hierarchy. Surely such a clear, intuitive interface such as Google’s must be

more usable. Fortunately for librarians, the research in each of these studies

indicates that users still want what libraries do best: A variety of services, tools,

and help for overwhelmed and uninformed users. The key to usability is clear

labeling, logical structure, and the right information at the point of need (Dee &

Allen, 2006, p. 71). It is important that, in keeping with libraries’ mission to

educate and inform, librarians make the most of their library Web sites.

As Duncan and Holliday (2008) observed, “to many users, the Web site is

the library,” warranting the same planning and attention to structure a physical

building would require (p. 302). Library Web sites will only be useful as long as

users can locate what they are trying to find. This means the information must

be organized and labeled according to the way the user thinks. Web librarians

must involve the user in the design, and confirm their designs with usability

testing.


Bibliography

Augustine, S., & Greene, C. (2002). Discovering how students search a library web site: A usability case study. College & Research Libraries, 63(4), 354-365.

Abstract: “Have Internet search engines influenced the way students search library Web pages? The results of this usability study reveal that students consistently and frequently use the library Web site's internal search engine to find information rather than navigating through pages. If students are searching rather than navigating, library Web page designers must make metadata and powerful search engines priorities. The study also shows that students have difficulty interpreting library terminology, experience confusion discerning difference amongst library resources, and prefer to seek human assistance when encountering problems online. These findings imply that library Web sites have not alleviated some of the basic and long-range problems that have challenged librarians in the past. Reprinted by permission of the publisher. “ Annotation: This study has been heavily cited in library Web site usability literature because it was one of the first to make several key observations. Using usability testing techniques in the form of tasks, the authors, librarians at the University of Illinois library, noted that student participants demonstrated a preference for Google- and Yahoo-type searching because they found library Web site navigation and organization confusing. A detailed qualitative analysis of the results is provided. Search Strategy: I selected ERIC because of its coverage of academic

libraries. I chose a keyword search because this was my initial search. I chose to limit my search to articles in English, and to keep the size of the set manageable, I limited the search year to 2000-present. I reasoned that the most relevant literature on the topic would be within the last 11-12 years. I also limited the set to not include document types of books or chapters.

Database: ERIC [Dialog] Method of Searching: Keyword searching Search String: ss (usability or information()architecture) and librar?

s s1 not (dt=book or dt=books or dt=book chapter)

s s2/eng


s s3 and py=>2000

Scholarly/Refereed Status: This article was available in the online databases of

ERIC as well as LLIS, and was listed when “peer-reviewed” was selected. The journal is also listed as peer-reviewed and scholarly by Ulrich’s. According to Web of Science, College & Research Libraries has a moderate impact factor (0.683) when compared to other library scholarly journals.

Chen, Y-H., Germain, C. A., & Rorissa, A. (2011). Defining usability: How

library practice differs from published research. Portal: Libraries and the Academy, 11(2), 599-628.

Abstract: “Library/information science professionals need a clearly articulated definition of usability/Web usability to implement intuitive websites. In this study, the authors analyzed usability definitions provided by the ARL library professionals and those found in the library/information science and computer science-information systems literature. Quantitative and qualitative methods were used to identify similarities and differences between the attributes emphasized by the two sets of definitions based on information behavior models and human-computer interaction (HCI) frameworks. Results indicated that both groups overlooked critical usability elements, such as environment and information objects/content/resources. Thus, the authors proposed a working, multi-faceted definition that presents a holistic view of usability. “ Annotation: Chen and his colleagues offered a definition of Web usability for librarians to improve consistency and effectiveness of library Web sites. The authors found that librarians’ definitions needed additional attributes and also needed different priorities for their requirements. Among these were the attributes for user satisfaction (which would translate into a repeat customer) and content usefulness. A thorough description of the survey and subsequent analysis gives authority to the research. The authors are librarians at the University at Albany, State University of New York, NY. Search Strategy: I selected ERIC because of its coverage of academic

libraries. I chose a keyword search because this was my initial search. I chose to limit my search to articles in English, and to keep the size of the set manageable, I limited the search year to 2000-present. I reasoned that the most relevant literature on the topic would be within the last 11-12 years. I also limited the set to not include document types of books or chapters. I also verified Chen’s authority by conducting an author search in Web


of Science. According to Web of Science, Chen has written at least 5 other articles related to this topic and digital libraries since 2004.



s s2/eng

s s3 and py=>2000

Scholarly/Refereed Status: This article was available in the ERIC online database

and was listed when “peer-reviewed” was selected. The journal is also listed as peer-reviewed and scholarly by Ulrich’s. A search in Web of Science’s Journal Citation Index shows that this journal has a moderate impact factor (0.870) as compared with other library and information science journals.

Dee, C., & Allen, M. (2011). A survey of the usability of digital reference services

on academic health science library web sites. Journal of Academic Librarianship, 32(1), 69-78.

Abstract: “Reference interactions with patrons in a digital library environment using digital reference services (DRS) has become widespread. However, such services in many libraries appear to be underutilized. A study surveying the ease and convenience of such services for patrons in over 100 academic health science library Web sites suggests that cumbersome access and difficulty of use may be a key restraining force. “ Annotation: This study looked at the usability of digital reference services using a combination of surveys, tasks and traffic logs from a proxy server. The authors noted that users’ perceptions of ease of use of a system often differed from reality. Clear terminology was critical to how many clicks it took to find online reference services, regardless of whether a link was placed on the site homepage. This study adds credibility to Jeng’s (2005) findings about the number of clicks the user makes being directly tied to user satisfaction. A weakness of the study was that students already familiar with library terminology participated. It is unknown how successful non-library users would be. Cheryl Dee is an Assistant Professor at the School of Library and Information Science and Maryellen Allen


is an Assistant Librarian at Tampa Library, both at the University of South Florida, Tampa. Search Strategy: I selected ERIC because of its coverage of academic




s s2/eng

s s3 and py=>2000


and was listed when “peer-reviewed” was selected. The journal is also listed as peer-reviewed and scholarly by Ulrich’s. According to Web of Science, Journal of Academic Librarianship has a moderate impact factor (0.870) when compared to other library scholarly journals.


Dickstein, R., & Mills, V. (2000). Usability testing at the University of Arizona Library: How to let the users in on the design. Information Technology and Libraries, 19(3), 144-151.

Abstract: “Reports the work carried out at Arizona University Library to redesign SABIO, the Library's information gateway, with particular reference to the extensive usability testing that was carried out to ensure that the Web site catered for user needs to the greatest extent possible. The activities of Access 2000, the project team, in using the students to guide them through the redesign of the Web site to overcome some of the usability problems of the earlier system, are described. The elements identified as needing attention included: making links predictable; making screens as simple as possible; using graphics minimally; and making pages easy to scan. “ Annotation: Although card sorting has been used as part of Web usability since the mid-1990s to develop user-centered site organization, this is one of the earliest articles to include the process for a library Web site. Dickstein and Mills, librarians at the University of Arizona Library, Tucson, used the technique in the process of making their library Web site user-focused, resisting the urge to be librarian-focused. This exercise was combined with heuristic evaluation and task-based usability testing. A weakness of the results was the lack of details regarding the card sort analysis. Search Strategy: I used Web of Science’s advanced search function and

searched on the TI field. I restricted my search to English language articles, 1997-2012 and searched only the Science Citation and Social Science Citation Indexes. I also performed a subsequent search in Web of Science in the Cited Reference tab on Dickstein r* in the author field and 2000 in the publication field, and found the article cited in 29 other publications.

Database: Web of Science (online) Method of Searching: Title search Search String: TI=(information architecture or usability) AND librar*

Scholarly/Refereed Status: Information Technology and Libraries is listed by

Ulrich’s as peer-reviewed and scholarly. According to the journal’s Web site, the journal “is a refereed journal published quarterly by the Library and Information Technology Association (LITA), a


division of the American Library Association” (http://ejournals.bc.edu/ojs/index.php/ital/about/editorialPolicies#peerReviewProcess). According to the Journal Citation Index, this journal has a relatively moderate impact factor (0.528).

Duncan, J., & Holliday, W. (2008). The role of information architecture in

designing a third-generation web site. College & Research Libraries, 69(4), 301-318.

Abstract: “Library Web sites have evolved over the past decade, from simple pages with a few links to complex sites that provide direct access to hundreds of different resources. In many cases, this evolution occurs with little overall planning, often resulting in Web sites that are hard to manage and difficult for users to navigate. This article outlines the process of using Information Architecture (IA) to redesign a third-generation library Web site from the ground up. The result was a much more usable and cohesive library Web site that meets the needs of a broad range of users. “ Annotation: Like Turnbow, et al. (2005) and Dickstein and Mills (2000), Duncan and Holliday, librarians at the Merrill-Cazier Library at Utah State University, addressed the organizational issue of their library Web site, combining a closed card sort with a follow-up task-based usability test. The authors identified taxonomy as being critical to findability. They noted that single, discrete categories may not be the best way to serve users’ needs, and librarians should consider multiple paths to the same information in site navigation. A thorough description of the blueprint and testing process was included. Search Strategy: I selected Library Literature & Information Science

because of its coverage of both libraries and scientific research. I chose a keyword search because this was part of my initial search. I first performed the search separately, then as part of a multi-file search in Dialog with ERIC, removing the duplicates.

Database: Library Literature & Information Science [Dialog] Method of Searching: Keyword searching Search String: ss (usability or information()architecture) and librar?



s s2/eng

s s3 and py=>2000

Scholarly/Refereed Status: This article was available in Library Literature and

Information Science, but was not included in my initial search in ERIC. College & Research Libraries is listed by Ulrich’s as peer-reviewed and scholarly. The article also come up in the online database for Library Literature and Information Science, when “usability” is included in the title field and “duncan” is included in the author field, while “peer-reviewed” is checked.

Gullikson, S., Blades, R., Bragdon, M., McKibbon, S., Sparling, M., & Toms, E. G.

(1999). The impact of information architecture on academic web site usability. Electronic Library, 17(5), 293-304.

Abstract: “Recent studies of web-site use indicate that people do not come to the web for an ‘experience’, they come for information. Yet, to date, web-site design has been synonymous with the ‘look and feel’ of a site overlooking the significance of a site’s information architecture. In this study, we assessed the effect of the information architecture of an academic web site: how information is categorised, labelled and presented, and how navigation and access are facilitated. Twenty-four participants from six faculties attempted to answer typical questions often asked within an academic milieu. They were able to find the answers to just over half the questions successfully and, in subjective assessments, gave the site a failing grade. We address how the information architecture affected their ability to negotiate the site and, additionally, make recommendations for the key ingredients: information design, access tools, and navigational aids. “ Annotation: This article is often cited in the literature because it is one of the first to address the idea that how information is structured in a Web site is as important to the appearance of the site. Though the authors were from the School of Library and Information Studies at Dalhousie University, Halifax, Nova Scotia, Canada, the focus of the study was an academic Web site. This article appeared in a library-focused journal and subsequently became the foundation for future in-library usability studies. Search Strategy: This article was cited in many other articles I reviewed

for this project, including Augustine, Duncan, Swanson and McGillis. I found it through reference chasing in the


Augustine article, and I discovered it was also in the Duncan article, then in McGillis. I noticed it in many others as well. My initial search string in ERIC and Library and Information Science through Dialog did not pull up this article at first because I restricted my search set to articles published in 2000 or later. I believed that the notion of information architecture as it pertained to library Web sites was a more recent development. It appears, however, that this topic was a ground breaking one in library literature as early as 1999.

Database: N/A Method of Searching: Footnote chasing Search String: Referenced in:

Duncan and Holliday (2008)

Augustine and Greene (2002)

McGillis (2001)

Swanson (2001)

Scholarly/Refereed Status: The journal The Electronic Library is listed by Ulrich’s

as both scholarly and peer-reviewed. This article also came up in a search of the online database, Library Literature & Information Science Full Text when “peer-reviewed” was checked.

Jeng, J. (2005). Usability assessment of academic digital libraries: Effectiveness,

efficiency, satisfaction, and learnability. Libri, 55, 96-121. Abstract: “This study is to develop and evaluate methods and instruments for assessing the usability of digital libraries. It discusses the dimensions of usability, what methods have been applied in evaluating usability of digital libraries, their applicability, and criteria. It is found in the study that there exists an interlocking relationship among effectiveness, efficiency, and satisfaction. It provides operational criteria for effectiveness, efficiency, satisfaction, and learnability. It discovers users' criteria on ‘ease of use,’ ‘organization of information,’ ‘terminology and labeling,’ ‘visual attractiveness,’ and ‘mistake recovery.’ Common causes of ‘user lostness’ were found. ‘Click cost’ was examined.“


Annotation: This study looked at how to design usability testing for academic digital libraries, and specifically looked at the attributes of efficiency, effectiveness, satisfaction and learnability. Efficiency was measured in terms of the amount of time and number of steps required to retrieve information; effectiveness was a measure of whether the user was successful and whether mistakes were made; satisfaction addressed ease of use, labeling appearance, and organization. The study then looked at how these attributes affected learnability. Using statistical analysis, which is described in detail, the author found that there was a strong correlation between effectiveness and the number of clicks, and there was also a strong correlation between effectiveness and satisfaction. Judy Jeng has written other articles on the same theme, also often cited, and is currently on the editorial board of the Journal of Web Librarianship (http://www.lib.jmu.edu/org/jwl/). Search Strategy: I found this article in Web of Science as I was looking up

cited references from my ERIC and LLIS search through Dialog. Though the journal Libri may not have a high impact factor as compared with JASIST (when Journal Citation Index was used), I found when I searched for jeng j in Web of Science under “cited reference search” that this article has been cited in 13 other articles.

Database: Web of Science Method of Searching: Forward Citation Search String: An article by Janet Chisman had come up in my search of

ERIC in Dialog. In “cited reference search” in Web of Science, I typed chisman j* in the cited author field and 1999 in the cited year. I selected both entries for the author in both Journal of Library Administration and Library & Information Science Research entries. Among the results of citing articles was this article. I also found that this article appeared when I looked up augustine s and mcgillis l in Web of Science in the “cited reference search.”

Scholarly/Refereed Status: This journal is listed by Ulrich’s as peer-reviewed and

scholarly. The Web site for Libri also indicates papers are refereed and scholarly.


King, H. J., & Jannik, C. M. (2005). Redesigning for usability: Information architecture and usability testing for Georgia Tech Library’s website. OCLC Systems and Services, 21(3), 235-243.

Abstract: “Purpose: To describe the efforts by the Georgia Institute of Technology Library to keep its website relevant and current by incorporating user testing, both outsourced and in-house. Design/methodology/approach: A chronological history of the Library’s web presence with a discussion of how user testing was conducted with a vendor and how future testing will be conducted by the Library itself. Findings: Illustrates issues the Georgia Institute of Technology Library has faced in regards to its website in the past, and future plans to test the site to maintain currency and strive to meet he users’ needs. Originality/value: This paper offers practical information for undertaking usability testing of a library website.“ Annotation: As noted by Maloney and Bracke (2004), libraries are facing increasing challenges as they attempt to combine under one “roof” a variety of complex systems. These systems occupy an “information space” (King and Jannick, 2005, p. 235) in which users must complete their tasks. The authors, who represent the Georgia Institute of Technology Library and Information Center in Atlanta, Georgia, used a two-phased approach to improve usability of their Web site: A content reorganization based on peer-reviews and user surveys, and a task-based usability test conducted by a professional firm. A permanent, in-library usability lab was created so that the library could perform its own usability testing in the future. One weakness of this article is that the details of the usability tests and results are not included. Search Strategy: I selected Library Literature & Information Science




s s2/eng

s s3 and py=>2000


Scholarly/Refereed Status: This article was available in Library Literature and Information Science, but was not included in my initial search in ERIC. OCLC Systems and Services is listed by Ulrich’s as peer-reviewed and scholarly. The article also come up in the online database for Library Literature and Information Science, when “usability” is included in the title field and “king” is included in the author field, while “peer-reviewed” is checked.

Lewis, K. M., & Hepburn, P. (2009). Open card sorting and factor analysis: A

usability case study. Electronic Library, 28(3), 401-416. Abstract: “Purpose: The purpose of this paper is to describe the process, analysis, results, and implications of a card sorting usability study conducted during the planning stages of a web site redesign project at the University of Illinois at Chicago. Design/methodology/approach – The methodology was based on recommendations from usability literature. An open card sort was conducted with 14 students and one faculty member using 93 cards labeled with content from the library's web site. The subjects were asked to “think aloud” and explain their rationale for sorting the cards. The researchers used statistical analysis software to run a factor analysis on the results. Findings – The researchers extracted 11 categories of cards that loaded together and 27 cards that did not fit a category. The categories showed evidence of clustering by shared words, format, and process or task. Cards that did not load were standalone categories, or were redundant or meaningless to the subjects. Research limitations/implications – The open card sort methodology and large number of cards resulted in cumbersome data that required specialized analysis. The qualitative data were critical to the interpretation of the quantitative data. Practical implications – Libraries can use the process and analysis as a model for their own card sort usability studies. Results can be used to inform the naming of content and the creation of library web site architecture. Originality/value – This study is unique in its use of the open card sort technique and factor analysis of the results. The results illustrate users' perceptions of library terminology and web site structure.“ Annotation: The authors used an open card sort in conjunction with a survey and a follow-up usability test to determine an ideal structure for their Web site. Because of the large number of cards needed for their site due to the complexity of its content, the authors rejected traditional analyses of “eyeballing” results, spreadsheet analysis, and cluster analysis. Instead, the authors used a factor analysis that allowed for multiple categorizations of content and uncovered the genuine thought processes of users looking for information, capturing


terminology that made the most sense to them. The article provides a thorough description of the results and methods of this analysis. Krystal Lewis is a librarian at Temple University, Philadelphia, PA, and Peter Hepburn is a librarian at the University of Illinois, Chicago, IL. Search Strategy: I selected Library Literature & Information Science




s s2/eng

s s3 and py=>2000


Information Science, but was not included in my initial search in ERIC. The Electronic Library is listed by Ulrich’s as peer-reviewed and scholarly. The article also came up in the online database for Library Literature and Information Science, when “usability” is included in the title field and “lewis” is included in the author field, while “peer-reviewed” is checked.


McGillis, L., & Toms, E. G. (2001). Usability of the academic library web site: Implications for design. College & Research Libraries, 62(4), 355-367.

Abstract: “Today's savvy library users are starting to equate the library Web site with the physical library. As they accomplish, virtually, many personal activities such as online shopping, banking, and news reading, they transfer those experiences to other activities in their lives. This increases their expectations about the functionality of a library Web site and how one interacts with it. The purpose of this study was twofold: to assess the usability of an academic library Web site and to better understand how faculty and students complete typical tasks using one. Thirty-three typical users successfully completed 75 percent of a set of typical tasks in about two minutes per task and were satisfied with the clarity and organization of the site. Despite their success in completing the tasks, however, they experienced difficulties in knowing where to start and with the site's information architecture—in particular, with interpreting the categories and their labels. The authors concluded that library Web sites fail to take into account how people approach the information problem and often reflect traditional library structures. Reprinted by permission of the publisher. “ Annotation: McGillis, a public services librarian at Sir Wilfred Grenfell College, and Toms, associate professor for Faculty of Information Studies at the University of Toronto, make the point that library Web sites that follow the physical library structure model in structuring their content are unusable. The authors also observed that terminology must be clear and intuitive, free from library “jargon.” With this article, a real shift began to take place in which library Web developers sought to think like users, and not like librarians. Toms has written other scholarly articles on the topic of Information Architecture, including the seminal work she co-authored with Gullikson (1999). Search Strategy: I selected ERIC because of its coverage of academic





s s2/eng

s s3 and py=>2000


and was listed when “peer-reviewed” was selected. The journal is also listed as peer-reviewed and scholarly by Ulrich’s.

McHale, N. (2008). Toward a user-centered academic library home page. Journal

of Web Librarianship, 2(2-3), 138-176. Abstract: “In the past decade, academic libraries have struggled with the design of an effective library home page. Since librarians' mental models of information architecture differ from those of their patrons, usability assessments are necessary in designing a user-centered home page. This study details a usability sequence of card sort and paper and online survey methods conducted at the Auraria Library, which serves the University of Colorado Denver, the Metropolitan State College of Denver, and the Community College of Denver. The three top complaints about the existing Web page included information overload, lack of visual cues and guidance, and difficulty getting to what students knew was the heart of the library's Web page: online resources. Using qualitative and quantitative data from the card sorting sessions, the Web librarian, under the direction of the Web Advisory Committee, was able to create a more user-centered home page. Unique to this study are the use of undergraduate students in creating test instruments and a means of gathering information about what students value most in their library's home page. “ Annotation: The author contends, uniquely, that a library home page, as the point of entry to a site, should be as successful in getting the user to his goals as a search in Google or Yahoo. This would directly support the use of tools and services that the library provided. Questionnaires and a card sort were used for this purpose. Instead of library staff designing the test, however, the test was designed and administered by members of the target population: Undergraduate students. The results were all the more user-focused, as far as the students were concerned. More work will need to be done to include faculty into the process. Nina McHale is Web Librarian for the Auraria Library in Denver, CO, and is currently an editor for the Journal of Web Librarianship. Search Strategy: I selected ERIC because of its coverage of academic

libraries. I chose a keyword search because this was my initial search. I chose to limit my search to articles in English, and to keep the size of the set manageable, I limited the search year to 2000-present. I reasoned that


the most relevant literature on the topic would be within the last 11-12 years. I also limited the set to not include document types of books or chapters.



s s2/eng

s s3 and py=>2000

Scholarly/Refereed Status: The Journal of Web Librarianship is listed by Ulrich’s as

both scholarly and peer-reviewed. According to the journal’s Web site, “the Journal of Web Librarianship is an international, peer-reviewed journal” (http://www.lib.jmu.edu/org/jwl/).

Maloney, K., & Bracke, P. J. (2004). Beyond information architecture: A systems

integration approach to web-site design. Information Technology and Libraries, 23(4), 145-152.

Abstract: “Users' needs and expectations regarding access to information have fundamentally changed, creating a disconnect between how users expect to use a library Web site and how the site was designed. At the same time, library technical infrastructures include legacy systems that were not designed for the Web environment. The authors propose a framework that combines elements of information architecture with approaches to incremental system design and implementation. The framework allows for the development of a Web site that is responsive to changing user needs, while recognizing the need for libraries to adopt a cost-effective approach to implementation and maintenance. “ Annotation: Maloney, Director of Technology at the University of Arizona Libraries, Tucson, and Bracke, head of Systems and Networking at the Arizona Health Sciences Library, Tucson, take a systems approach in applying concepts of information architecture to Web site usability. The authors noted that many library Web sites have not been structured to help users achieve their goals. In this sense Google is attractive because it is seamless and does not require prior familiarity in order to use it successfully. The authors’ expertise in the area of information architecture, though technical, is communicated in clear illustrations.


Search Strategy: I found this article in Web of Science as I was “reference

chasing” articles from my ERIC and LLIS search through Dialog. After I had developed a list of articles that I deemed highly likely to support my topic, I began looking through references at the end of several papers. Though I found this article in Cockrell, I ended up not using that article for this bibliography.

Database: N/A Method of Searching: Footnote chasing Search String: I found this article referenced in another by Cockrell and

Anderson which was one that was found in my initial Dialog search of ERIC and LLIS. The article was:

Cockrell, B.J., & Anderson, E. (2002) How do I find an article? Insights from a Web usability study. Journal of Academic Librarianship, 28(3), 122-132.

Scholarly/Refereed Status: The journal, Information Technology and Libraries, is

listed by Ulrich’s as both scholarly and peer-reviewed. This article also came up in a search of the online database, Library Literature & Information Science Full Text, when “peer-reviewed” was checked.

Robbins, L. P., Esposito, L, Kretz, C., & Aloi, M. (2007) What a user wants:

Redesigning a library’s web site based on a card-sort analysis. Journal of Web Librarianship, 1(4), 3-25.

Abstract: “Web site usability concerns anyone with a Web site to maintain. Libraries, however, are often the biggest offenders in terms of usability. In our efforts to provide users with everything they need for research, we often overwhelm them with sites that are confusing in structure, difficult to navigate, and weighed down with jargon. Dowling College Library recently completed a redesign of its Web site based upon the concept of usability. For smaller libraries in particular, such a project can be a challenge. The Web site is often maintained by one or two people, and finding the time and resources to conduct a usability study is difficult in that situation. Additional demands of a site redesign, from restructuring page layouts to adding visual appeal, only add to the burden. However, our team of four librarians was able to do it. We focused on


vocabulary and organizational structure using a card-sort analysis. This analysis taught us how our users approach the information on our site. Task-based testing confirmed what the card-sort analysis had taught us and smoothed out design problems. Incorporating user feedback at nearly every stage of the process allowed us to create a site that more closely mirrors how our users look for information on our site. This study details how using testing and analyzing results throughout the redesign process created a better, more user-friendly Web site. “ Annotation: The card sort exercise was the focal point of the user-centered redesign on the authors’ Web site for this article. The authors, all librarians from Dowling College Library, Oakdale, NY, recognized that the key to findability was in discovering how their users searched for information. They identified that the best way to find this out was to conduct a card sort. Their analysis of their results using statistics to determine correlations gave their findings validity. When results proved to be widely varying, they were tempted to impose their librarian expertise on nomenclature. Subsequent usability testing showed that users were confused by librarian jargon. A weakness of the study is the lack of raw results. The authors summarize without giving numbers in their Excel spreadsheet. Data collection implements are included. Search Strategy: While trying to find full-text articles published in the

Journal of Web Librarianship, which Drexel does not subscribe to, I began browsing ECO to find out who owned the journal. That led me to browse the journal article titles in ECO.

Database: ECO through FirstSearch Method of Searching: Browsing Search String: I entered Journal of Web Librarianship in the search

field with ECO as the source and began browsing all of the 537 results. I selected the unique items that were not already found in my Dialog search of ERIC and LLIS.




Robins, D., & Kelsey, S. (2002). Analysis of web-based information architecture in a university library: Navigating for known items. Information Technology and Libraries, 21(4), 158-169.

Abstract: “Presents a descriptive study of the Louisiana State University Libraries' Web site. The aim of the study was to gain some idea of user demographics and satisfaction with the site at a given point in time and to test the site's navigation system. The goal was to find out who was using the site, why they were using it, and to what extent they were satisfied with the site's navigation. Tasks were assigned (searching for known items) to subjects to better determine the extent to which the site's navigation system facilitated locating information on the site. Evaluation of the navigation system was based on a ratio of correct clicks to the sum of incorrect and back button clicks. This ratio may be compared to some predetermined optimal number of clicks needed to retrieve a known item. The implications of this research are both theoretical and practical. These models of in-house, Web-based information seeking may be used by other institutions of a similar nature that seek to provide useful Web sites for their users as well as to provide a basis for further research on the problem of Web-based development of information retrieval systems. “ Annotation: Robins, Assistant Professor at the University of Pittsburgh School of Information Sciences, and Kelsey, Electronic Reference Services and Web Development Coordinator at Louisiana State University libraries, Baton Rouge, looked at user satisfaction [one of the attributes mentioned by Chen, et al. (2011) as not commonly embraced by librarians] in an evaluation of Louisiana State University Library’s Web site. One of the key points made was that the right information appearing at the right moment to accomplish a task was critical to user satisfaction. The test implements and results are included, as are thorough descriptions of the analyses. Search Strategy: I found this article in Web of Science as I was looking up

cited references from my ERIC and LLIS search through Dialog.

Database: Web of Science Method of Searching: Forward Citation Search String: An article by Jerilyn Veldof that looked promising had

come up in my search of ERIC in Dialog. In “cited reference search” in Web of Science, I typed velodf j* in the cited author field and 2001 in the cited year. I selected both entries for the author in both Journal of


Library Administration and Library & Information Science Research. Among the results of citing articles was this article.

Scholarly/Refereed Status: This journal is listed by Ulrich’s as peer-reviewed and

scholarly. The journal’s Web site also indicated that it is peer-reviewed and scholarly.

Swanson, T. (2001). From creating web pages to creating web sites: The use of

information architecture for library web site redesign. Internet Reference Services Quarterly, 6(1), 1-12.

Abstract: “The librarians at the Moraine Valley Community College redesigned the library’s Web site using Louis Rosenfeld and Peter Morville’s application of information architecture to Web site design. The librarians at Moraine Valley Community College learned that successful Web site design hinges on matching the needs of the user group with available technology. Web site designers must be able to create a flexible organizational scheme that allows a balance to be struck between competing factors such as hierarchical depth versus breadth, site structure versus site flexibility, and usable labels versus needed jargon. Most importantly for librarians operating in the online environment, Web designers must understand that users visit Web sites to access information, but it is a site’s information architecture that makes the site successful.“ Annotation: Swanson, Teaching and Learning Librarian at Moraine Valley Community College, Palos Hills, IL, is one of the early proponents in the literature of using information architecture as part of a redesign of a library Web site. Like Jerilyn Veldof (2001), he saw that ease of use of a site and information architecture are linked. The author used information architecture best practices in redesigning a site for his library. These best practices included creating a plan as a guide. Research tools included surveys, usage statistics, and interviews. Anticipating the topic of the article he wrote 10 years later, he concluded that Web site design is more than just a look or an obvious search engine. One weakness of the study is that details of the user survey and its analysis are not included. Search Strategy: I selected Library Literature & Information Science





s s2/eng

s s3 and py=>2000


Information Science, but was not included in my initial search in ERIC. Internet Reference Services Quarterly is listed by Ulrich’s as peer-reviewed and scholarly. The article also came up in the online database for Library Literature and Information Science, when “information architecture” is included in the title field and “swanson” is included in the author field, while “peer-reviewed” is checked.

Swanson, T., & Green, J. (2011). Why we are not Google: Lessons from a library

web site usability study. Journal of Academic Librarianship, 37(3), 222-229. Abstract: “In the Fall of 2009, the Moraine Valley Community College Library, using guidelines developed by Jakob Nielsen, conducted a usability study to determine how students were using the library Web site and to inform the redesign of the Web site. The authors found that Moraine Valley's current gateway design was a more effective access point to library resources than a mock-up site which incorporated a central-search box on the site homepage. This finding stands in contrast to the observed trends of library Web site design that emphasizes a ‘Googlized’ search. “ Annotation: This study was a recent addition to the foundation of research done by Augustine and Greene (2002) and McHale (2008). The question addressed is whether a Google-type search prominently featured on a library’s home page is a viable alternative to a logically architected, usable site. The resounding answer confirmed in this paper is “No.” Swanson and Green reiterated the point the Augustine and Greene made that resources are only valuable if they are findable from a library home page (2002, p. 227). Unlike Swanson’s earlier study, the authors include detailed descriptions of the study design and results. Swanson and Green are librarians at Moraine Valley Community College, Palo Hills, IL.


Search Strategy: I selected ERIC because of its coverage of academic libraries. I chose a keyword search because this was my initial search. I chose to limit my search to articles in English, and to keep the size of the set manageable, I limited the search year to 2000-present. I reasoned that the most relevant literature on the topic would be within the last 11-12 years. I also limited the set to not include document types of books or chapters.



s s2/eng

s s3 and py=>2000

Scholarly/Refereed Status: This article was available in the online databases of

ERIC as well as LLIS, and was listed when “peer-reviewed” was selected. The journal is also listed as peer-reviewed and scholarly by Ulrich’s.


Turnbow, D., Kasianovitz, K., Snyder, L., Gilbert, D., & Yamamoto, D. (2005). Usability testing for web redesign: A UCLA case study. OCLC Systems & Services, 21(3), 226-234.

Abstract: “The purpose of this paper is to describe the processes the UCLA Library Website Redesign Team used to develop a new library website responsive to the needs of the broad population of UCLA Library users. Using a combination of structured analyses of the previous library website, user surveys, a card sort protocol and a think-aloud protocol, the Website Redesign Team procured sufficient information to meet its redesign goals: established clear site organization and navigation, utilized user-centered nomenclature, ensured easy access from the library homepage to relevant information, developed a unified institutional visual identity throughout the site, and enabled a content management system. Standard usability methods such as surveys and the card sort and think-aloud protocols are essential tools for evaluating and redesigning complex multi-layered websites. Since the redesign process is not finite, these tools contribute to keeping a website current and responsive to the needs of its users. This case study provides an example that the Redesign Team hope will empower readers with tools and knowledge that they can use to perform similar tasks in their own environment. “ Annotation: The authors, representing various libraries at the University of California, Los Angeles, California, demonstrate the importance of including card sorting in the overall process of structuring a site for maximum usability. As with other studies, the card sort is followed by a task-based usability test. These authors also used surveys and created a spreadsheet to document the current environment. Unlike others who have used card sorting tests, the authors provided definitions for terms used to describe the content they were sorting. These became essential to participants in organizing and labeling the content of the cards used in the card sort exercise. The authors could then make informed decisions about the design of the site, including what kinds of templates would be needed and what consistent navigation would look like. Search Strategy: I selected Library Literature & Information Science


Database: Library Literature & Information Science [Dialog] Method of Searching: Keyword searching


Search String: ss (usability or information()architecture) and librar?


s s2/eng

s s3 and py=>2000


Information Science, but was not included in my initial search in ERIC. OCLC Systems and Services is listed by Ulrich’s as peer-reviewed and scholarly. The article also came up in the online database for Library Literature and Information Science, when “usability” is included in the title field and “turnbow” is included in the author field, while “peer-reviewed” is checked.

Veldof, J. R., & Nackerud, S. (2001). Do you have the right stuff? Seven areas of

expertise for successful web site design in libraries. Internet Reference Services Quarterly, 6(1), 13-38.

Abstract: “In the mid-1990s it was common for one person in the library's technology department to design or redesign the library's Web site but today it is virtually impossible for one person, or even two or three, to create a user-centered Web site design given the complexity of the Web environment and users' needs. Explores the seven areas of expertise now necessary to create successful library Web sites: project management; information architecture; usability; access for people with disabilities; graphic design; content creation; and programming. Sets out the authors' collective experience in designing nine different library and university Web sites and in testing the usability of four other library Web products. “ Annotation: By 2001, the concept of information architecture was beginning to become a common consideration in the design of library Web sites. The authors are the only ones to observe that librarians are uniquely poised to make good information architects, thanks to their knowledge of information science. Librarians often categorize, classify, organize according to hierarchy, index, and utilize labeling systems. These are also the skills of an information architect. Applying these skills to Web site design will mean nothing, however, if the user is not part of the process. Learning the needs of the user can be done through focus groups and interviews. Decisions can then be made following usability testing. Veldof is User Education Coordinator and Shane Nackerud is Web Services Coordinator for the University of Minnesota Libraries-Twin Cities, MN.


Veldof is the author of several articles on Web usability and many articles on library instructional design. Search Strategy: I used First Search to search the ECO database. I chose

this database because of its coverage of both education and library science topics.

Database: First Search – OCLC Electronic Collections Online Method of Searching: Combination of Keyword and Subject searching Search String: Using advanced search, I used the following criteria:

Usability (keyword)

OR

“information architecture” (keyword)

Limit to (subject): Libraries and Library Science [Z 662-Z 1000.5]

Year: 2000-2012

Scholarly/Refereed Status: The journal is listed as peer-reviewed and scholarly

by Ulrich’s. It also states on its Web site that the journal is scholarly and peer-reviewed.

Whang, M. (2007). Card-sorting usability tests of the WMU libraries’ web site.

Journal of Web Librarianship, 2(2-3), 205-217. Abstract: “This article describes the card-sorting techniques used by several academic libraries, reports and discusses the results of card-sorting usability tests of the Western Michigan University Libraries’ Web site, and reveals how the WMU libraries incorporated the findings into a new Web site redesign, setting the design direction early on. The article briefly describes open and closed card-sorting techniques and quantitative and qualitative methods of analyzing data commonly used in computer and library science fields. Findings from this study allowed the library design team to vastly improve its initial redesign decisions for a new Web site tabbed navigation system. Card sorting not only helped the design team validate its redesign decisions, but it also opened the WMU libraries to outside innovation, inviting Web site visitors to redesign the Web site free from the libraries’ influence. The simple and inexpensive techniques used here may be useful to any Web librarian or design team embarking upon redesign and usability testing of their own sites and interested in building a more compelling, insightful Web site.“


Annotation: The author, Head of Web and Internet Services at Western Michigan University Libraries, Kalamazoo, MI, demonstrated that card sorting exercises can follow task-based usability testing and still be valid, as long as it is conducted early in the design process. He also describes the appropriateness of qualitative versus quantitative analysis of the results as determined by set size, something not addressed in other card sorting articles. In his study, Whang used a series of surveys to validate the results of his card sort. He described the process and reasoning behind decisions made regarding analysis. The validation surveys are included in the study. Search Strategy: I selected ERIC because of its coverage of academic




s s2/eng

s s3 and py=>2000



Conclusion and Personal Statement

I found this assignment challenging, but rewarding. Challenges included

winnowing large numbers of search results to the most relevant, important


articles, designing searches that retrieved relevant results without being too

constrictive, organizing my results in a way that I could use them, and

synthesizing the studies into a cohesive report.

My initial searches done before learning about Dialog were largely

ineffective because I was mistakenly ANDing terms instead of ORing them.

Once I changed my search, I retrieved articles on card sorting, which is

commonly included in discussions of usability testing, but is really a tool to get

at site structure and link labels. The term “information architecture” often

doesn’t appear in these articles, despite its relevance.

My biggest challenge was in deciding where to start. I had sorted my

articles by date. Many articles had similar titles, making it hard to select one title

that stood out among the rest. I chose the Duncan and Holliday (2008) article as

my starting point because of the highly relevant title and its position as near the

middle of my date range. I began by scanning the content and references of the

article. It wasn’t long before I noticed the same author names appearing over

and over again. It was also helpful to read what authors were saying about each

other in the literature review sections of articles to give me an idea of why an

author was cited. This was also helpful in narrowing down my final

bibliographical entries.

My biggest reward in this assignment was in learning so much about this

topic. I learned how to put the YOU in usability. As I learn the skills of

librarianship, I will remember how I initially approached library systems and

tools with confusion. I hope I will be able to expand this research in future

classes.

I certify that:

• This assignment is entirely my own work.


• I have not quoted the words of any other person from a printed source or website without indicating what has been quoted and providing an appropriate citation.

• I have not submitted this assignment to satisfy the requirements of any other course.

Signature Janette Dauenhauer Date 3/16/12

library website usability: how a library’s site content is...

Documents