Library Services within the Total User Experience: Discovery and Digital Scholarship

William MischoGrainger Engineering Library Information CenterUniversity of Illinois at Urbana-Champaign

National Institute of Technology SilcharJanuary 21, 2015

Library Services within the Total User ExperienceLibrary Services within the Total User Experience

Core Library Services• Activities that support the campus research and

instructional experience– Discovery gateway systems that offer

“smart” search assistance that mimics the actions of a trained reference librarian

– Dynamic digital environments for active engagement with students and faculty

• The role of the academic library; the importance of establishing key campus and external partnerships

Library Services within the Total User Experience

Role of the Library• Core values: library’s changing roles in

collection development, discovery and delivery, and digital scholarship

• Library’s role in end-to-end lifecycle of knowledge creation and knowledge management

• Concept of local collection much less meaningful

• Re-imagineering the library

Discovery and Delivery• Libraries at a Crossroads• Roger Schonfeld (Ithaka): “this is a good moment for

academic libraries to step back to reconfirm (or reconsider) their vision for discovery, to ensure that their visions connect with information-seeking practices and preferences…”

• Exploring three technologies:– Federated search– Web-Scale Discovery Systems (WSDS)– Hybrid Bento-style approach

Full Library Discovery• Overarching issue is how to optimize access within

the complex information landscape we live in: online catalogs, publisher full-text, institutional repositories, A&I services, local and multi-institutional digital content, learning resources, etc.

• Full Library Discovery – Lorcan Dempsey• Not only collections but services – subject area

specialists, library webpages, smart search assistance, pathfinders, *personalization, etc.

Search & Discovery• Gateway function and full-text is King• Federated search: broadcast and merge over remote

information targets• Web-Scale Discovery Services: single aggregated

index harvested from content holders• Hybrid bento-style federated search where the screen

display is partitioned into tile areas• Numerous libraries investigating Bento

Search & Discovery• University of Illinois at Urbana-Champaign (UIUC)

Library discovery efforts:– Easy Search federated search and

recommender system– WSDS: Primo Central, 2011-2014; WorldCat

Discovery, 2013-; Ebsco Discovery Service API– Bento-style search and display

Areas of Focus:--Search Assistance--Information Content Integration--Full Library Discovery

OPACs-UIUC VuFind-State VuFind-Voyager-WorldCat Discovery-

Easy Search APIDirect Links (Best Bets)Libguide MatchesSpelling SuggestionsSubject Area IdentificationAuthor SuggestsLimit/Broaden SuggestsAdded Title Search linksJournal AtoZ MatchesDOI matchesJournal & Article Locator

Discovery and Delivery EnvironmentTotal User Experience Focus

Full-Text Linking:

-- CrossRef and DOI

-- Local Link resolver

-- A&I via OpenURL local Link Resolver

-- Google Scholar and Link Resolver

Users“I am looking for a specific item” “I need articles, books about X”

Federated Search (broadcast), Web-Scale Discovery (aggregation)Hybrid Bento-style Search/Display

Subject Area Identification-Open Worldcat Class Label-Recommended targets-Departmental library-Librarian subject specialist

Scopus API

EbscoEDS API

Dark Target Searches

Google Discovery Services APIs

Link ResolverA to Z Search

LibraryWebsite Search

A&I article databasesAPIs

Direct Links

Transaction Logs

Information targetHTTP Connectors

Digital Content Management

Repository

Course Management

Systems

Multiple Publisher Repositoriese-journals, e-books

National/International Digital PlatformsDPLA, Hathi, LOCKKS, CrossRef,

DataCite (datasets)

Web-Scale Discovery Systems

University of Illinois Library Gateway• Revised Gateway portal introduced in September

2007, powered by Easy Search federated search system:oRecommender and discovery systemo Employs context-specific and adaptive Search

Assistance mechanismsoHelps with search strategy modification and

navigationo Takes users into native interfaces at point of

completed searchoWrites out custom transaction logs

Evidence-Based Services• We believe in user testing and incorporating

user needs in systems• User behavior studies and custom transaction

log data and user survey input • Online Catalog user studies have been

contradictory• Increasing usage of Gateway as access

mechanism for known-item search

10

UIUC Library Gateway

UIUC Library Gateway

UIUC Library – Bento Display

UIUC Library – Bento Display

Digital Scholarship Centers in Libraries• Support growing & widespread e-research and digital

scholarship• Bring together expensive technologies for use by all

campus departments• Bring together expertise to serve multiple campus

departments• Support graduate and undergraduate students who

desire to create digital projects but do not have access to tools and expertise in their department

Library/Campus Partnership• Library as place, Place as service• Collaboration, e-scholarship, innovation, experiment

with new forms of scholarship and publishing • Knowledge visualization, data visualization, data

analytics, next-gen design, user interaction• Lifecycle of scholarly content. Helping the institution

become a learning organization

Digital Scholarship Center facilities: UCLA

Visualization facility at Hunt Library NCSU

Brown University Library

Brown University Library

Georgia State University Library

Grainger Engineering Library Information Center: Illinois

Data Management Issues• U.S. federal grant agencies require Data

Management Plans: NSF, NIH, DOE• The 2013 White House OSTP mandate and

White House memo• Need to demonstrate importance to campus of

a campus-wide infrastructure for data management

• Multiple campus stakeholders: library, academic computing center, research offices, colleges, departments

UIUC Library Initiative factors• There is a growing number of disciplinary and

government repositories• We wanted to lead the efforts for a campus-

wide program• Wanted to show upcoming compliance issues,

dangers of current practices• Approached offices of: Vice-Chancellor for

Research and Provost

NSF Data Management Plans• Data Management Plans (DMPs): required

element in NSF proposals, January 2011

• July 2011: the Library, working with the campus Office of Sponsored Programs and Research Administration (OSPRA) began an analysis ofDMPs in submitted NSF grant proposals

• Currently, looked at 1,600 grants with 1,260 in the analysis.

Reasons for DMPs• Make research data available and sharable • Use of data for verification of results and

reproducibility of research• Federal agency can show significant return on

investment to justify funding• Important to know storage venues and

mechanisms for sharing and reuse• Use of local templates and local campus

resources such as institutional repository

Reasons for Analysis

•What storage venues and mechanisms for sharing and reuse are being used?

•Are the PI’s using local templates and local campus resources such as the IDEALS?

•Is the campus in danger of losing data? Of losing grants because of compliance issues?

Analysis• Analysis attempts to characterize and classify

DMPs into categories

• DMPs assigned multiple categories

• 1,260 DMPs from July 2011 to November 2013

Data Venue and Risk

Data LocationSubmitted Proposals Funded Proposals

Risk of Loss/Corruption/ Breach

n=1260 n=298

PI Server/Website 64% High 61% HighDepartmental Server/Website

11.2%Medium to

High7%

Medium to High

Campus-Wide Resource 52.9%

Low45%

LowIDEALS (Institutional Repos.)

21.9% 19.8%

NCSA 4.3% 16.4%Disciplinary Repository/Cloud

25.8%Medium to

Low21.4%

Medium to Low

Remote Repository 28%Medium to

High22.8%

Medium to High

Optical Disk, Specimens, Analog

19.4% Out of Scope 11% Out of Scope

Analysis

• Any differences in storage venue or technologies between the unfunded proposals and the funded proposals?

• Any differences between the proposals from the first year and the more current proposals?

• Other differences in proposal categories between funded and unfunded

• 734 active NSF awards, $861.8 million

Implications and Conclusions1. No significant differences between

funded/unfunded proposals in storage venues - no advantage in specifying IDEALS or Disciplinary

2. More recent proposals suggest IDEALS and disciplinary repositories included at a significantly higher level

• Questions remain: what is the role of the library? The campus? The subject discipline?

Linking of Data and Article Literature

• The standards are CrossRef DOIs and DataCite DOIs: we need bi-directional links

• Role of publishers in data storage and linking• Linking is of interest to publishers, authors, grant

agancies• Potential for a Linked Open Data application• Potential for data repository recommender

service

Linking of Data and Article Literature

• Example is search of Scopus to retrieve an article by an author or on a topic

• The article DOI (CrossRef DOI) is extracted and a link to DataCite using this CrossRef DOI is constructed

• The matches at DataCite are displayed along with their links from the DataCite DOI to the dataset results landing page/producer page

Follow-on• Vice-Chancellor for research and Provost have

funded a campus-wide Research Data Service - RDS

• Assist in compliance with federal agencies and assist researchers with DMPs, DOIs, storage mechanisms both short-term and preservation

• Develop important partnerships with campus units and national entities

Illinois Environmental Scan• Subscribe to 200+ specialized article and report

databases• 65K online journals• Collection budget of $15 million• Online is, in many cases, the exclusive access

point• Have a legacy online catalog (search over

metadata, holdings of books, journals, +)• We operate in a distributed, heterogeneous

information environment

35

Evidence-Based Services• We believe in user testing and incorporating

user needs in systems• User behavior studies and custom transaction

log data and user survey input • Online Catalog user studies have been

contradictory• Increasing usage of Gateway as access

mechanism for known-item search

36

We Do Tools and Services• Debate over role of Library in “discovery”• Desire for one-stop-shopping• We look at custom transaction logs over Gateway and

ES Advisory Committee• Easy Search recommender and search assistance tool• Known-item access important• Problem with blended displays and full-text searching

37

Today’s Library Gateways• Academic libraries are moving from federated

search systems to web-scale discovery systems (WDS) with centralized aggregated indexing to Bento-style hybrid search and discovery

• WDS feature single-entry search boxes with faceted limiting operations’ is faceting useful?

• As we transition to WDS & Bento, understanding and supporting user search behaviors is important

Retrieval Issues • Little use of advanced systems; facet use is

moderate, about 30%• Full-text search retrieval and relevancy

ranking issues• Large number of known-item cut-and

paste searches

39

Web-Scale Discovery Features• Hosted and in the networked Cloud• Aggregated index of licensed publisher-based

bibliographic content – journal articles, newspaper articles, book chapters, dissertations, and local OPAC and digital content

• Single entry search box• Blended display format with limiting facet

operations & other Web 2.0 features• Central aggregated index contrasts with the

federated search/broadcast search approach

40

Why are Libraries Embracing WDS?• Rival to Google; hope to bring back users• Extend the OPAC and integrate local

digitized content & Institutional Repository• Negative experiences with federated

search/broadcast search• One-stop shopping approach attractive• Next Big Thing – keep up with others

41

The Big Issues • Understanding and supporting user search

behaviors is important • The effectiveness of full-text search and

relevancy ranking is under question• Some modules (e.g. OPAC) may be better

than central index search/discovery • Important to distinguish between

discovery features and known-item access mechanisms

42

Easy Search Custom Transaction Log• We record user actions, system suggestions, search

reformulations, and clickthroughs• 2010-2011 study of 1.4 million searches and 1.5 million

clickthroughs • 2014 study of 1 million searches, 1.1 million clickthroughs

over 10 month period May 2013 to March 2014• Initially supported by NSF and IMLS

• OPAC studies ambiguous results; web search engine behaviors different

Our Study of Custom Transaction Log Data • Conducted an evidence-based study of user

behavior in the Easy Search University of Illinois Gateway

• Looked at 1.4 million searches and 1.5 million clickthroughs performed by users in Fall 2010 and Spring 2011 semesters; now updating

• Our transaction log studies were supported by National Science Foundation and Institute of Museum and Library Studies

UIUC Library Gateway

UIUC Library Gateway

UIUC Library – Bento Display

UIUC Library – Bento Display

Examples • Competent Jerks, Lovable Fools, and the Formation of Social

Networks• The virtual supermarket: An innovative research tool to study

consumer food purchasing• Mothers and fathers : a study of the development and

negotiation of parental behaviour• Matouschek, Kellis, Serrano, Fersht Nature• Kinsella and Phillips 2005• Mowen, A. J., Kyle, G. T Jackowski, M. (2007). Citizen preferences

for the corporate sponsorship of public sector park and recreation organizations. Journal of Nonprofit and Public Sector Marketing, 18(2), 93

• Hemenway, D. (2010). Why We don't Spend Enough on Public Health. New England Journal of Medicine, 362(18), 1657

• university, need for money• J. Quant. Spectrosc. Radiat. Transfer.

2014 Transaction Log Analysis • Breakdown of searches from the main Gateway:

– Easy Search and associated tabs: 902,420 76.5% • the gateway Easy Search default tab: (55.1%)

– advanced search: 56,170 4.8%– departmental library page searches: 158,750 13.5%– myeasysearch: 18,159 1.5%– search suggestion searches: 42,708 3.6%

• Total: 1178207

50

Search Assistance Features• Context-specific suggestions and

prompts for:– Spelling changes – Redo as author search– Direct links to frequent or special search terms

or Libguides– Links to matching e-journal title(s)– Links to citation linker for full-text retrieval– Limit results to exact phrase, title word(s), or

title phrase– Dark background searches displayed if needed

2014 Transaction Log Analysis • Gateway searches total: 902,420

– Easy Search tab: 649,320– Books search tab: 106,375– Article tab: 76,375– Journal Title tab: 70,350

• 2007: 49.4% of all searches were known item• 2011: 51.2% of searches in 54% of sessions were

known-item

52

2014 Transaction Log Analysis • Sample of 974,137 user-entered searches:

– average words per query is 5.11 words– This is up from earlier studies:

• 3.58 words per query in 2008; 3.76 words per query in 2009; 4.33 words per query in 2011

• 93,642 one word, 158,054 greater than 8 words• 49.6% were 3 words or less, 29.4% are 6 words or more• longest search was 299 words

53

2014 Transaction Log Analysis • 20.98% of all searches in 29.13% of all sessions

used a search assistance suggestion or custom added search result link

• 57.2% of the 489,272 search sessions were single query sessions

• 6% of the sessions contain 6 or more queries, 1,157 sessions contain more than 20 queries, and 96 search sessions contain more than 50 queries

54

Primo Clickthrough Study May-June 2014 • Primo clicks 15,068 times between 05-01-2014 and

06-21-2014. Total of 147,326 clickthroughs during that time

• Looked at a sample of 478 of the 15,068 searches• 245 known-item searches (51.8%) and 228 topical

searches (48.2%)• Of the 245 known-item searches, 159 were judged

successful - they brought back the expected results in the Primo first page or as the top item

55

Primo Clickthrough Study (2)• Of the 159 ‘successful’ searches, all of them were also

successful in Scopus, Ebsco, CrossRef, WorldCat Discovery, Web of Knowledge, VuFind, IShare, Google, or Google Scholar searches

• Removing the Google results left 9 searches that were successful in Primo but not via Scopus, Ebsco, Web of Science, CrossRef, VuFind, IShare, or WorldCat Discovery. Removing CrossRef and WorldCat Discovery resulted in 35 searches successful in Primo but not successful in Scopus, Ebsco, Web of Science, VuFind, or IShare

56

Primo Clickthrough Study (3)• Found that CrossRef and WorldCat Discovery

good at retrieving successful search results for full citation searches

• 48 known-item searches that were unsuccessful in Primo but successful in one of the other Easy Search targets - Scopus, Ebsco, Web of Science, CrossRef, VuFind, IShare, WorldCat Discovery, or Google/GS

57

Primo Clickthrough Study (4)• Evaluated sample against Arizona State’s

Summon. In one case, a known-item search that was unsuccessful in Primo yielded a first page match in the AZU Summon. However, there were 50 known-item searches that were successful in our Primo that were unsuccessful (item not on top or first page) in the AZU Summon

58

Easy Search and Bento approach user survey• 120 survey results that users of Easy Search

provided in October 2014• 43 were daily users and 53 indicated they were

weekly users• 99 surveys where users provided us with

comments on their Easy Search experiences. In 77, users made either a very favorable (28) or favorable comment about Easy Search

59

Easy Search and Bento survey (2)• 91 surveys with comments about the Bento

approach with 66 of these providing very favorable or favorable comments

• 40 users with Very favorable/favorable views of ES who also held Very Favorable/Favorable views of the Bento search

• 17 user surveys where ES was regarded as Very Favorable/Favorable and the Bento version was regarded as Unfavorable or Neutral

60

Easy Search and Bento survey (3)• 21 surveys where ES was regarded as

Unfavorable or Neutral and the Bento version as Very favorable/Favorable

• Overall, this shows a significant acceptance of the Bento search and display approach

• Note that one of the Bento options is to re-perform the user search in “classic” Easy Search

61

Bento Table• Table comparing the feature sets of 17 Bento

search/display implementations deployed by university libraries, including our version

• Number of Bento versions that do not do spell checking and do not provide direct links from the bento display page to full-text

• None of the others employ the one-click to full-text PDF that we do

62