user-generated content and social discovery in the academic library catalogue: findings from user...
DESCRIPTION
Co-presented with Martha Whitehead at Access 2008TRANSCRIPT
1
User-Generated Content and Social Discovery in the Academic Library Catalogue:
Findings From User ResearchMartha Whitehead, Queen’s University Library
Steve Toub, BiblioCommons
Access 2008, Hamilton, October 2, 2008
2
Agenda
1 Overview of the problems we want to address2 Observations about research and intriguing ideas
to explore3 User research on user-generated content and
social discovery at an academic library4 Preliminary thoughts on how to ensure quality5 Vision and status of BiblioCommons academic
product at Queens6 Questions
3
The Problem
Discovery
getting answers to questions you don’t know how to ask
finding gems you don’t know exist
4
5
Observation: Learning and Research are Social
Social
considering the judgements
and insights of others
6
User-Generated Content
Narrow sensetags, ratings,
reviews
Broader sensecurated content
7
Existing Implementations: UGC in the narrow sense
8
Existing Implementations: UGC in the broader sense
9
Existing Implementations: UGC in the broader sense
10
Existing Implementations: UGC in the broader sense
11
Existing Implementations: UGC in broader sense
12
What have scholars told us about research?
Ontario Council of University Libraries (OCUL) Scholars Portal User Study, May 2008, with Usability Matters
Primary Objective:Understand the information research processes of experienced researchers in a variety of disciplines
Methodology6 collaborative design sessions with 8-10 participants each3 discipline areas
Arts & HumanitiesSocial SciencesSciences (Natural, Applied, Health, etc.)
13
Discover, gather, create, share
A Multi-Dimensional Framework for Academic Support, June 2006, University of Minnesota Libraries funded by Mellon Foundation*, building upon John Unsworth’s concept of scholarly primitives: “basic functions common to scholarly activity across disciplines, over time, and independent of theoretical orientation.”**
Validated in OCUL study: A number of groups introduced the terms “interaction”, “collaboration” or “conversation” and ultimately decided that these are not discrete steps in the process but, rather, are overarching throughout all “phases”. They were adamant about the importance of this aspect of their information research process.
* http://www.lib.umn.edu/about/mellon/UMN_Multi-dimensional_Framework_Final_Report.pdf**John Unsworth. “Scholarly Primitives: What Methods do Humanities Researchers Have in Common and How Might Our Tools Reflect This?”
Humanities Computing, Formal Methods, Experimental Practice Symposium, Kings College, London, May 13, 2000. http://jefferson.village.virginia.edu/~jmu2m/Kings.5-00/primitives.html
14
Aspects of “typical” research process
Thinking of keyword search termsBrainstorm with colleaguesUse a familiar or seminal article or journalUse books and bibliographies/references
Participants appreciate being able to find sources based on:What others searched forMost accessed resourcesMost cited resourcesMost credible or prestigious sources
In storyboarding the ideal discovery process, they included: Seeing recommendations from “authorities”Classics in the fieldWays to find “surprises”, unanticipated sources
15
Social?
The research process is social, but that doesn’t refer to social tools
Few had heard of Del.icio.usEveryone had heard of Facebook but only one or
two were using it for academic purposes
16
“Social discovery,” not “social networking”
User consultation at Queen’s University, February 2007, with Usability Matters
4 focus groups, 6-8 participants each, undergrads, grads, facultyObjectives
Elicit opinions regarding current state of the library website(s)Determine how the library website can support user needsExplore user expectations around information searching
and browsingDetermine user interest and expectations around library
instruction and guidanceElicit ideas for new website features and services
Personalized website featuresSocial networking features
17
Research on UGC: Attitudes
18
19
20
The main point
Participants expressed their desireto know what trusted colleagues (professors, fellow researchers) think
21
March 2008: Research on the research process
At Queen’s, held in a library conference roomFaculty, graduate students and undergraduates
Recruitment of faculty through emailRecruitment of others through link/survey on website, catalogueHour-long one-on-onesStarting with library website, existing catalogue
Also got reaction to several NGCs
4 faculty; 2 staff researchers (both taught occasionally); 2 graduate students; 5 undergraduates
Librarians2 focus groups of 8-10 librarians each
22
March 2008: Key findings on the research process (1)
Faculty, graduate students, and undergraduatesEmploy post-search limiting very infrequently
Only people who’ve seen facets before noticed facets in NGCsReformulate queries via back button and typing variant terms
insteadDo not reformulate queries by ORing all terms or using
truncation, wildcards
LibrariansWant the power of a comprehensive command-line query syntax
(e.g., fielded searches, Boolean operators, truncation)
23
March 2008: key findings on the research process (2)
Faculty, graduate students, and undergraduatesAvoid hyperlinked subject headings
Result screens that list headings are perceived as confusing and unhelpful
Virtual shelf-browsing tested well
LibrariansValue the bibliographic control provided by subject headings
24
March 2008: key findings on the research process (3)
Faculty, graduate students, and undergraduatesDon’t experience pain when manually managing and formatting
citations
LibrariansThink DirectExport to RefWorks is high priority
25
March 2008: key findings on the research process (4)
Faculty, graduate students, and undergraduatesWhen frustrated, students tend not to as for help, especially
from librariansEven in the face of atrocious information literacy skillsInformation literacy skills not dependent on age/role
LibrariansWant to help
26
June 2008 one-on-ones devoted to UGC
Recruited May 28-30 via Facebook ad that linked to SurveyMonkey survey
175,662 impressions. 111 clicks = 0.06% click-through rate
36 completed surveys = 32% who viewed survey completed it
Nine one-hour sessions June 4, 5 at BiblioCommonsEight undergraduates (U of T, Queens, Western)
3 completed fourth year2 completed second year1 completed first year
Two graduate students: brother-sister pair who attended the same hour session
27
Solicited feedback using a variety of different media
• Web sites they’ve used to make product choices (e.g., Restaurantica.com, Rotten Tomatoes)
• Their university library web page and OPAC• And a page in University of Michigan’s OPAC directing
their attention to MTagger• On-screen mockups of three different ways
in which they could contribute UGC in the library context
• Paper mockup of a course-related hub page that provided links to various items associated with a course
• List of possible motivations for contributing UGC
28
This round focused more on narrow UGC than broad
Existing application already has strong support for curated listsCompleted initial exploratory R&D on broad UGC in public library productWill need more R&D to tweak list creation flows
Faculty syllabus tool that outputs to course reserves, bookstore, courseware
Convert ad hoc reference interview into a BiblioCommons list
Still early days for narrow UGC in academic library catalogueNot sure why data elements and motivations apply in academic contextNot aware of any published research on this
Concepts and motivations apply equally well
29
Attitudes about UGC
30
Attitudes about UGC
31
Tagging?
What is this?“I’m not entirely sure… I would like to assume… some sort of user feedback…but I don’t why they’d say “tag” … but if you were a student there you’d probably know what it was…”
D_____, graduated 4th year, entering FIS
32
Tagging!
Have you ever heard of tagging?“No.”Have you ever tagged on Facebook?
“Of course. That’s with photos. They have it for text – but no one uses it.”D____, Graduated 4th year, entering FIS
33
34
35
The most important data elements (course context)
Relevance to courseA lot of “Is it going to be on the test?” But some nuances we could explore further:
“How related is this reading is to other readings?” or “How related this reading is to the lecture?”
Clarity [a.k.a. level of difficulty] was second most popular data element
36
Mechanics of data entry process for UGC
Most said they might not fill out more than 1-2 data elements.The “sliders” represented in the mockups tested well.
Moving a “slider” is perceived to be easier than typing. If there are no open-ended comments that involve typing, it may be
possible to ask for more than two data elements using “sliders”. Segments on the “sliders” should be visually apparent; could be 5-or
10-point scale.
Anonymity, i.e., having the ability to choose a username that isn’t personally identifiable, will make contributions more likely.
Most wanted other students to view their comments.Even the person least likely to contribute (when we first saw the concept,
her reaction was, “Why would I do that?”) in the end said she would be willing to share comments with others if she only had to fill in 1-2 things for each item and if her comments were anonymous.
37
38
Course hub mockup
39
Reaction to the course hub mockup
“That would be amazing!”
“That would become my new first place to go to start my searches.”
--L_____, completed 2nd year at Western
40
Likeliest opportunities to contribute
The course reading context seems the likeliest opportunity to contribute.Solicitation in this context only works if the syllabus were online and the student is
looking at the syllabus online rather than a paper copy.When using the syllabus online, if they were looking at what to read for Week 2,
they wouldn’t mind an invitation to comment on the readings for Week 1.They would like the ability to edit their comments later on.
Soliciting contributions from a “recently returned” also well received.Mixed reaction on email solicitations on “recently returned”.
Need to probe further on how to make emails palatable. Several said they didn’t want email at all. One person said that if she’d much prefer email but not on each recently returned item but only if she got a single email once a month.
The “answers” mockup may only be utilized by who would ask questions, a definite minority.
Many fiercely resist the idea of asking others for help, even a TA or professor.
41
Possible motivations to contributeEarn Campus Credits
Chances to win prizes$ off fines, bookstore, foodservices “Printer Points”Charitable contributions
Opportunity to give feedback / Have my say.which online articles, library materials or course readings are useful; which are not
To the libraryTo my professors
Contribute / Give back to my university…the library. Help build a richer, more useful catalogue / database.
Get recommendations, suggestions – for materials I might not have otherwise found
Help others/everyone get to useful resources faster. More time thinking – less time finding
Quid pro Quo: I earn rights to ask others questions, when I answer some myself.
42
Possible motivations to contributeEarn Campus Credits
Chances to win prizes$ off fines, bookstore, foodservices “Printer Points”Charitable contributions
Opportunity to give feedback / Have my say.which online articles, library materials or course readings are useful; which are not
To the libraryTo my professors
Contribute / Give back to my university…the library. Help build a richer, more useful catalogue / database.
Get recommendations, suggestions – for materials I might not have otherwise found
Help others/everyone get to useful resources faster. More time thinking – less time finding
Quid pro Quo: I earn rights to ask others questions, when I answer some myself.
43
The #1 motivation
Helps [others] get to useful resources fasterHelp us be more helpful to youCould do more probing on languageSeveral commented that they liked the idea of spending
“less time finding, more time writing”
Strong sense ofPay it forward
“If I do it now, it will help others later” “If others do it, it will help me when I need it”
Empty restaurant syndromeSome fears of being the first to contribute
if they did not see evidence that others were doing the sameStronger indication they’d contribute
if they saw that everyone else was doing it
44
It seems pretty easy to “buy” student participation
Even the one student who had consistently said she wouldn’t be likely to contribute quickly checked off all 4 “Campus Credit” concepts as motivating.
45
Primary barriers to contributing
Many (but not all) are unwilling to support freeloaders.
However, they do like being able to freeload themselves and do see the connection that someone must contribute for others to freeload.
Worried about being accused of plagiarism makes students reluctant to share with peers.
Course-related sharing may need to be sanctioned by the professor of that course to allay these fears.
46
Strategies for ensuring quality
Patron authentication: fosters more measured and insightful commentsIt is not possible to leave anonymous comments in our systemEvery username is associated back to a real user authenticated against the library
database
Aggregation: helps you not draw conclusions from a single sourceAbility to see all reviews by particular reviewerAbility to view reviews on an item by many different customers.
Design a marketplace of ideas: self-managed system, not editorial reviewCapture positive/negative sentiment in a structured wayReview the reviewsExpose more attributes than binary hasReview vs. lacksReview
“3 students in your class found this reading helpful to prepare for the midterm exam”
“4 faculty included this item in course reading lists”
47
Vision for BiblioCommons’ academic product
Provide a smart catalogue that offers an outstanding user experienceWhen searching, for most of the people, most of the time, less is moreConditional display of request actions, no buttons leading to a dead endSurface curated content: lists, course reading lists, student bibliographies, etc.Provide ways to get answers to questions that users didn’t know how to ask
Organize by courses, assignments; not LCSH or broad subject guidesEnsure UGC data elements are course-centric, assignment-centricProvide course hubs that unify access to disparate course-related links
Break down artificial barriers between data silosLibrary web siteElectronic resource A-Z listsCourseware (e.g., WebCT)Article databases
48
Our research and development with Queen’s
Priorities grounded in user researchSignificant new development for BiblioCommons2008/2009: Iterative beta release process
49
Questions?
Martha WhiteheadAssociate University Librarian, Queen’s University [email protected]
Steve ToubProduct Manager, Academic Services, [email protected]