are we getting warmer? query clarification in virtual reference marie l. radford lynn silipigni...
Post on 27-Mar-2015
215 Views
Preview:
TRANSCRIPT
ARE WE GETTING WARMER? QUERY CLARIFICATION IN VIRTUAL REFERENCE
Marie L. Radford Lynn Silipigni Connaway
Library Research Round TableALA Annual ConferenceWashington, DC
June 23, 2007
2
Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, & Librarian Perspectives
Funded by Institute of Museum & Library Services Rutgers University & OCLC (10/05-9/07)
Four phases:I. Focus group interviewsII. Analysis of 850 QuestionPoint live chat
transcriptsIII. 600 online surveysIV. 300 telephone interviews
3
Would you clarify?
L# Time Message 1 5:55 U Hello. I need help finding
information on the following: where in baltimore, maryland can I find peppermint oil, if a 15 year old can start diving classes now, and can a 15 year old get a office job at this age?
4
A helpful answer?
2 5:55 L [A librarian will be with you in a minute.] 3 6:03 L [Librarian X – A librarian has joined the
session.] 4 6:38 L One moment please. 5 7:05 L I will see what I can find. 6 7:25 U alright 7 11:54 L After looking at a few web sites it seems
that beginning scuba diving classes start at age 12 to 16. It depends on how good a swimmer the person is. As a scuba diver myself I think the age range sounds right.
5
i don’t want scuba diving classesi want driving classes
8 12:18 U i don’t want scuba diving classes 9 12:27 U i want driving classes 10 15:21 L From the us department of labor site. The Fair
Labor Standards Act (FLSA)… 11 16:27 U So how do I go about findint a office job for my
15 year old daughter 12 16:44 U and also I want to now about
driving classes not scuba diving 14 18:36 U alright\ 15 06:13 [user has closed this session]
6
Do librarians clarify? Good news!
n=581
75%
22%
3%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Yes No No clarificationnecessary
7
When librarians clarify…
50% (217) asked the highly recommended follow-up question:
“Does this completely answer your question?”
n=434
8
What do librarians clarify?
Topic Background Verify Their Understanding Search History Type of Resource Extent/Depth Referral Corrected User Misunderstanding
9
Librarians ask about TOPIC
45% (195) sought TOPIC information
“Is there a specific animal that you're researching?”
n=434
10
Librarians ask about BACKGROUND
31% (133) sought BACKGROUND information
“When is your paper due?”
“Are you in the NYC area?”
n=434
11
Librarians VERIFY
20% (87) VERIFIED their understanding
“So you are looking for the reasons why the Japanese attacked Pearl Harbor?”
n=434
12
Librarians ask about SEARCH HISTORY
19% (83) sought SEARCH HISTORY information
“Ok, and have you checked their Web sites yet?”
n=434
13
Librarians ask about TYPE OF RESOURCE
11% (49) sought TYPE OF RESOURCE information
“Are you looking for books, articles, or online information?”
n=434
14
Librarians ask about EXTENT/DEPTH
11% (48) sought EXTENT/DEPTH information
“Were you looking for in-depth history? Or a summary?”
n=434
15
REFERRAL
4% (16) asked a REFERRAL question
“Do you want me to refer your question to your library?”
n=434
16
CORRECTED MISUNDERSTANDING
2% (7) CORRECTED MISUNDERSTANDING
“By the way, what I told you about the list being in strictly chronological order was not correct; I'm sorry.”
n=434
17
How about the Users?
Users offered information about: Topic Background Extent/Depth Correct Librarian’s Misunderstanding
18
Users Clarify TOPIC
52% (69) offered TOPIC information
“I am doing marketing reseach on marylands restaurant industry”
n=132
19
Users Clarify BACKGROUND
24% (32) offered BACKGROUND information
“I am in 7th grade GT Sciene and I am Doing a 2 year research project…”
n=132
20
Users Gave EXTENT/DEPTH
20% (26) offered EXTENT/DEPTH information
“the thing is i need it to be simple and not to wordy”
n=132
21
Users CORRECT Librarian’s Misunderstanding
16% (21) CORRECTED the librarian’s misunderstanding
“No. The form says I need the correct name for the position.”
22
Users VERIFY
14% (19) VERIFIED their understanding
“So the best thing is to find it on the online catalog… is that correct?”
n=132
23
Users Offer SEARCH HISTORY
14% (18) offered SEARCH HISTORY information
“I looked in the card catalogue and didn't see it there...”
n=132
24
Users provide TYPE OF RESOURCE
9% (12) offered TYPE OF RESOURCE information
“...And [not] a book, online though”
“Yes, but I’m looking for more historical and architectural resources” n=132
25
Users ask for REFERRAL
4% (5) asked for a REFERRAL
“Can you contact me with anyone from the […] Library”
n=132
26
When did librarians & users clarify?
Surprising finding…
Two different patterns of clarification!
Librarians clarified in beginning of interaction
Users clarified in middle of interaction
27
When did they ask for clarification?
51%
32% 33%
8%
18%
2%0%
10%
20%
30%
40%
50%
60%
Beginning/Beforesearching
Middle/During searching End/After searching
Pe
rce
nta
ge
of
tra
ns
cri
pts
Librarian
User
n=581
28
Types of clarifying questions… Open or closed?
Percentage of all questions asked by librarians:
66% (554) Closed Questions
34% (284) Open Questions
n=838
29
Closed or open: Is it clear?
“Can you tell me more about what you are looking for?” An invitation to talk, but still a closed
question?
“Can I ask what grade level this is for so I don’t give you something that is too technical or too easy?” “Ok” An unhelpful, closed response to an
ostensibly open question.
30
Imposed Query
“Imposed queries emanate from both formal and informal relationships between people”
(Gross, 1999, p. 501).
“For a history project, I must do a poster diagram of the evolution of pyramids from masatbas to step pyramids to straight-sided pyramids.”
31
“User identifies an information need and asks a question” (Gross, 1995, p. 237).
“I want to plan a camping and rock climbing trip to Pilot Mountain. Do you have any recommendations?”
Self-Generated Query
32
Imposed query vs. Self-generated query
46%
24%
1%
28%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Unknown Self-generated query Imposed Query -School assignment
Imposed Query -Work related(n=581)
(9)
(140)
(165)
(267)
33
What makes a difference for accuracy?
Analysis of Accuracy of Ready Reference questions (n=180)
Does question clarification make a difference?
Does asking a follow-up question make a difference?
YOU BE THE JUDGE…
34
YES! Clarifying & follow-up improve accuracy!
84
47
18 80
20
40
60
80
100
120
140
160
180
Clarified Follow-up
Correct
Incorrect
35
Future Directions
Continue to collect & analyze data Online surveys
Librarian survey completed Non-user and User surveys in progress
Telephone interviews 100 with Librarians completed 100 Users in progress 100 Non-users in progress
36
END NOTES
• This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives.
• Funded by IMLS, Rutgers University, & OCLC Online Computer Library Center, Inc.
• Special thanks to Jocelyn DeAngelis Williams, Patrick Confer, Timothy Dickey, David M. Dragos, Mary Anne Reilly, Susanna Sabolcsi-Boros, Julie Strange.
• Slides available at project web site: http://www.oclc.org/research/projects/synchronicity/
37
Questions & Comments
Lynn Silipigni Connaway, Ph.D. Email: connawal@oclc.org www.oclc.org/research/staff/connaway.htm
Marie L. Radford, Ph.D. Email: mradford@scils.rutgers.edu www.scils.rutgers.edu/~mradford
top related