unbundling the ils: deploying an e-commerce catalog search solution andrew pace & emily lynema...
DESCRIPTION
Digital Library Federation Forum, April 12, 2006 Unbundling the ILS: Deploying an e-commerce catalog search solution What ILS Catalogs Do Well… (liberally stolen from Roy Tennant) Inventory control: What and where Inventory control: What and where Known item searching Known item searching “To enable a person to find a book of which either is known: author, title, or subject.” - Charles Cutter, Rules for CatalogsTRANSCRIPT
Unbundling the ILS: Unbundling the ILS: Deploying an e-commerce Deploying an e-commerce
catalog search solutioncatalog search solution
Andrew Pace & Emily LynemaAndrew Pace & Emily LynemaNCSU LibrariesNCSU LibrariesApril 12, 2006April 12, 2006
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
What we will cover: Online catalog: the problemOnline catalog: the problem Brief environmental scanBrief environmental scan Implementation: team, timeline, Implementation: team, timeline,
technologytechnology DemoDemo Usability, statistical results, relevance Usability, statistical results, relevance
studystudy So what?So what? Future plansFuture plans
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
What ILS Catalogs Do Well…(liberally stolen from Roy Tennant)
Inventory control: What and whereInventory control: What and where Known item searchingKnown item searching
“To enable a person to find a book of which either is known: author, title, or subject.”
- Charles Cutter, Rules for Catalogs
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Any search other than known itemAny search other than known item Known item searchingKnown item searching Anything other than books and journalsAnything other than books and journals Logical groupings of results (e.g. FRBR)Logical groupings of results (e.g. FRBR) Faceted browsingFaceted browsing Relevance rankingRelevance ranking Sideways searching (suggestions, Sideways searching (suggestions,
expansion of searches and search expansion of searches and search targets)targets)
What ILS Catalogs Don’t do Well…(liberally stolen from Roy Tennant)
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Endeca purchase decision Lots of topical searches and poor Lots of topical searches and poor
subject accesssubject access– Keyword gives too many or too few Keyword gives too many or too few
results – leads to general distrustresults – leads to general distrust– Misunderstanding of authority headingsMisunderstanding of authority headings
No relevancy ranking of resultsNo relevancy ranking of results Needed more responsiveness (speed)Needed more responsiveness (speed)
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
NextGen Library Search ToolsThe Next Generation catalog: more than just a faceliftThe Next Generation catalog: more than just a facelift– RedLightGreen (RLG)RedLightGreen (RLG)– OCLC FictionfinderOCLC Fictionfinder– Vivisimo clustered search Vivisimo clustered search – Aquabrowser visual context Aquabrowser visual context – Endeca Guided NavigationEndeca Guided Navigation– Innovative Interfaces “OPAC Pro”Innovative Interfaces “OPAC Pro”– Ex Libris “Primo”Ex Libris “Primo”– Polaris, AJAX-Enabled OPACPolaris, AJAX-Enabled OPAC– SirsiDynix Enterprise Portal System, FASTSirsiDynix Enterprise Portal System, FAST– Talis, et alTalis, et alWeb ServicesWeb Services– OCLC Custom WorldcatOCLC Custom Worldcat– Georgia Pines and the Library 2.0 BandwagonGeorgia Pines and the Library 2.0 Bandwagon
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Implementation Team 7 representative team members7 representative team members– Andrew Pace, Information Technology, ChairAndrew Pace, Information Technology, Chair– Cindy Levine, Research and Information ServicesCindy Levine, Research and Information Services– Emily Lynema, Info. Tech., ex officio (tech lead)Emily Lynema, Info. Tech., ex officio (tech lead)– Erik Moore, Info. Tech., ex officio (ILS librarian)Erik Moore, Info. Tech., ex officio (ILS librarian)– Charley Pennell, Metadata and CatalogingCharley Pennell, Metadata and Cataloging– Shirley Rodgers, Information TechnologyShirley Rodgers, Information Technology– Tito Sierra, Digital Library InitiativesTito Sierra, Digital Library Initiatives
TimelineTimeline– License / negotiation: Spring 2005License / negotiation: Spring 2005– Acquire: Summer 2005Acquire: Summer 2005– Implementation: August 2005 – January 12, 2006Implementation: August 2005 – January 12, 2006
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Technical Overview Endeca ProFind co-exists with Endeca ProFind co-exists with
SirsiDynix Unicorn ILS and Web2 SirsiDynix Unicorn ILS and Web2 online catalog.online catalog.
Endeca indexes MARC records Endeca indexes MARC records exported from Unicorn.exported from Unicorn.
Index is refreshed nightly with Index is refreshed nightly with records added/updated during records added/updated during previous day.previous day.
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Endeca ProFind Overview
Raw MARC data
NCSU exports and reformats
Flat text files
Data Foundry
Parse text files Indices
Navigation Engine
NCSU Web Application
HTTP
Client browser
HTTP
Endeca ProFind
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Endeca ProFind Overview
Raw MARC data
NCSU exports and reformats
Flat text files
Data Foundry
Parse text files Indices
Navigation Engine
NCSU Web Application
HTTP
Client browser
HTTP
Offline - Nightly
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Endeca ProFind Overview
Raw MARC data
NCSU exports and reformats
Flat text files
Data Foundry
Parse text files Indices
Navigation Engine
NCSU Web Application
HTTP
Client browser
HTTP
Always Online
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Integrating Endeca Endeca doesn’t understand MARC data / MARC-8 Endeca doesn’t understand MARC data / MARC-8
character encoding – translate to UTF-8 text filescharacter encoding – translate to UTF-8 text files Each night a script updates the data indexed by Each night a script updates the data indexed by
Endeca:Endeca:– Exports updated or new MARC records from Unicorn.Exports updated or new MARC records from Unicorn.– Reformats and merges these records with those already Reformats and merges these records with those already
indexed.indexed.– Starts Endeca re-index – completely rebuilding index for Starts Endeca re-index – completely rebuilding index for
the catalog.the catalog. Process requires about 7 hours.Process requires about 7 hours. Retain Web2 OPAC for some functionalityRetain Web2 OPAC for some functionality
– Authority searching - known items and cross-referencesAuthority searching - known items and cross-references– Detailed record pages – how to make Endeca -> Web2 Detailed record pages – how to make Endeca -> Web2
link?link?
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Quick Demo http://catalog.lib.ncsu.eduhttp://catalog.lib.ncsu.edu
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Some User Reaction““This is absolutely the coolest thing I've seen all This is absolutely the coolest thing I've seen all
century.” century.” - Will Owen, Head of Systems (UNC Libraries)Will Owen, Head of Systems (UNC Libraries)
““Also, I'm really digging the new NCSU library catalog. Also, I'm really digging the new NCSU library catalog. Very nice." Very nice."
- Educause staff (non-librarian)- Educause staff (non-librarian)
““The new Endeca system is incredible. It would be The new Endeca system is incredible. It would be difficult to exaggerate how much better it is than our difficult to exaggerate how much better it is than our old online card catalog (and therefore that of most old online card catalog (and therefore that of most other universities). I've found myself searching the other universities). I've found myself searching the catalog just for fun, whereas before it was a chore to catalog just for fun, whereas before it was a chore to find what I needed.”find what I needed.”- NCSU Undergrad, Statistics- NCSU Undergrad, Statistics
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Some Search Statistics (March 2006)
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Searches by Search Key
74971
32776
135639872
58381141
0
20000
40000
60000
80000
Keyword ISBN Title Author Subject Multi-Field
Search Key
Req
uest
s
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Navigation by Dimensions
17939
8653
7451
13607
23291
20867
17720
44197
49931
6790
0 20000 40000 60000
Author
Language
Subject: Era
Subject: Region
Library
Format
Subject: Genre
Subject: Topic
LC Classification
Availability
Dim
ensi
on
Requests
Some Navigation Statistics (March 2006)
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Other interesting tidbits… (March 2006)
3.6% of all searches had spelling 3.6% of all searches had spelling corrected automaticallycorrected automatically
2.6% of all searches had alternate 2.6% of all searches had alternate spelling suggestionsspelling suggestions
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Usability Testing Trends I 10 undergraduate students10 undergraduate students
– 5 with Endeca catalog5 with Endeca catalog– 5 with old Web2 OPAC5 with old Web2 OPAC
Endeca performed as well as OPAC for known-Endeca performed as well as OPAC for known-item searchingitem searching– 89% Endeca tasks completed ‘easily’ (8/9)89% Endeca tasks completed ‘easily’ (8/9)– 71% OPAC tasks completed ‘easily’ (15/21)71% OPAC tasks completed ‘easily’ (15/21)
Endeca performs better than OPAC for topical Endeca performs better than OPAC for topical searchingsearching– 61% Endeca tasks completed ‘easily’ (19/31)61% Endeca tasks completed ‘easily’ (19/31)– 3% Endeca tasks completed as ‘hard’ (1/31)3% Endeca tasks completed as ‘hard’ (1/31)– 33% OPAC tasks completed ‘easily’ (13/39) 33% OPAC tasks completed ‘easily’ (13/39) – 26% OPAC tasks completed as ‘hard’ (10/39)26% OPAC tasks completed as ‘hard’ (10/39)
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Usability Testing Trends II Relevance *most* importantRelevance *most* important
– ““Once I scroll through a page, I get pretty discouraged about Once I scroll through a page, I get pretty discouraged about the results...” the results...”
Web2 OPAC participant looking for resources on cat healthWeb2 OPAC participant looking for resources on cat health ‘‘Keyword’ term less intuitive / trusted than ‘Subject’ Keyword’ term less intuitive / trusted than ‘Subject’
and ‘Title’and ‘Title’– ““[I used] Keyword in Title because that’s what I want the [I used] Keyword in Title because that’s what I want the
book to be mainly referring to. But I also could’ve went book to be mainly referring to. But I also could’ve went Keyword in Subject. But if I’d have went Keyword Anywhere it Keyword in Subject. But if I’d have went Keyword Anywhere it would have had too big of a field to look through.” would have had too big of a field to look through.”
Web2 OPAC participant looking for resources on gene therapyWeb2 OPAC participant looking for resources on gene therapy When found, dimensions seem intuitive and usefulWhen found, dimensions seem intuitive and useful ‘‘Did you mean’ seems intuitiveDid you mean’ seems intuitive Students don’t necessarily treat the catalog like Students don’t necessarily treat the catalog like
Google!Google!
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
A study in relevance Are search results in Endeca more Are search results in Endeca more
likely to be relevant to a user’s query likely to be relevant to a user’s query than search results in Web2 OPAC? than search results in Web2 OPAC?
100 topical user searches from 1 100 topical user searches from 1 month in fall 2005month in fall 2005
How many of top 5 results relevant?How many of top 5 results relevant?– 51% relevant in Web2 OPAC; 31 no hits 51% relevant in Web2 OPAC; 31 no hits – 69% relevant in Endeca catalog; 12 no 69% relevant in Endeca catalog; 12 no
hitshits
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Relevance defined Relevance ranking in Endeca – select Relevance ranking in Endeca – select
from a variety of modules and order from a variety of modules and order them based on importance.them based on importance.
Relevance most important in Keyword Relevance most important in Keyword Anywhere - searches all fields.Anywhere - searches all fields.
At NCSU…At NCSU…1.1. Original query term(s) (no thesaurus, Original query term(s) (no thesaurus,
stemming, spell correction)stemming, spell correction)2.2. Exact phrase matchExact phrase match3.3. Field ranking (Title higher than Author higher Field ranking (Title higher than Author higher
than Table of Contents)than Table of Contents)4.4. Number of fields that contain term(s) …Number of fields that contain term(s) …
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
So what? It’s still just a catalog The library systems puzzleThe library systems puzzle Reversal of fortuneReversal of fortune
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
The library system puzzle
Catalog
SerialsA&I / FT DBs
Web
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
The library system puzzle
Catalog
SerialsA&I / FT DBs
Web
Digital Repositories
ERM Systems
Guided Navigation
Legacy ILS
Metasearch
IR
GS
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Reversal of fortuneOLD SEARCH MODEL
NEW SEARCH MODEL
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Future Plans Ongoing tweaks:Ongoing tweaks:– Continued usability testingContinued usability testing– Relevance ranking algorithms & spell correction Relevance ranking algorithms & spell correction
thresholdsthresholds– Additional browsing optionsAdditional browsing options
Endeca 2.0 ideasEndeca 2.0 ideas– FRBR-ized displayFRBR-ized display– Discussions with OCLC regarding FAST (Faceted Access Discussions with OCLC regarding FAST (Faceted Access
to Subject Terms) and FRBRto Subject Terms) and FRBR– Patron-generated refinements (folksonomies?)Patron-generated refinements (folksonomies?)– Enrich records with supplemental Web Services content Enrich records with supplemental Web Services content
– more usable TOCs, book reviews, etc.– more usable TOCs, book reviews, etc.– The death of authority searching (?)The death of authority searching (?)– More integration with QuickSearch, other data More integration with QuickSearch, other data
repositories, and third-party discovery toolsrepositories, and third-party discovery tools
Digital Library Federation Forum, April 12, 2006Digital Library Federation Forum, April 12, 2006Unbundling the ILS: Deploying an e-commerce catalog search solutionUnbundling the ILS: Deploying an e-commerce catalog search solution
Thankshttp://www.lib.ncsu.edu/endecahttp://www.lib.ncsu.edu/endeca
Andrew Pace, Head, ITAndrew Pace, Head, [email protected][email protected]
Emily Lynema, Systems Librarian for Digital Emily Lynema, Systems Librarian for Digital ProjectsProjects