marie l. radford, ph.d., rutgers university [email protected]
DESCRIPTION
CSI Cyberspace: A Multiple Case Study Investigation of the Untimely Demise of Seven Virtual Reference Services. Marie L. Radford, Ph.D., Rutgers University [email protected] M. Kathleen Kern, MLS, University of Illinois at Urbana-Champaign [email protected] New Reference Research: - PowerPoint PPT PresentationTRANSCRIPT
CSI Cyberspace: A Multiple Case Study Investigation of the
Untimely Demise of Seven Virtual Reference Services
Marie L. Radford, Ph.D., Rutgers [email protected]
M. Kathleen Kern, MLS, University of Illinois at Urbana-Champaign
[email protected] New Reference Research: 11th Annual Reference Research Forum, ALA, Chicago June 25, 2005
Problem Statement
Number of chat reference services growing 3000+ chat & IM (Coffman & Arret, 2004)
Alive & Clicking! Many services are successful and viewed as
integral Clicked the Bucket!
However, a number of chat services have died untimely deaths.
Why?
Research Questions What were the deciding factors for ending these
services? Who was involved in making the decision to end the
services? How could these failures/false starts have been
prevented? What has been the impact of discontinuing these
services? What are the similarities and differences among the
cases? What lessons can be learned to ensure the success
of existing services or newly developing services?
Modus Operandi - Method
Multiple Case Study Method Selection of Cases & Informants
7 Cases Data Collection
Email Surveys Telephone Interviews Examination of Documents
Method Continued
Analysis of Results Case Descriptions of Each Case Comparison of Cross – Case Findings
Similarities Differences
Iterative Process Examination of Documents
Results Despite our original & advertised title
we had 7 cases not 6. 5 Academic 1 Public 1 Consortium
One surprise, all 7 agreed to be interviewed & share their experiences.
Another surprise – Most (5-7) had no grant funding!
Starting & Operating the Service – Common Themes
No additional staff hired. Broad target audience – same as for in-
person service. Distance education students a focus. Limited hours of operation.
Average 22.4 hours per week User response positive.
Postmortem - 5 Major Reasons for Discontinuing Services
Funding – Budget CutsLow UseStaffingTechnical Problems Institutional Culture
Reasons for Discontinuing Service
Low
Use
Funding Staffing Tech.
Prob.
Inst.
Cult
Primary 3 3 1 1 0
Note: Totals to 8 since Case B listed 2 primary factors (Technical Problems and Low Use).
Reasons for Discontinuing Service
Low
Use
Funding Staffing Tech.
Prob.
Inst.
Cult
Primary 3 3 1 1 0
Secondary 2 0 2 0 1
Total 5 3 3 1 1
CSI Cyberspace Mystery
Why did these services have such low volume?
Exhibit A - VolumeCase Monthly Qs % of total Qs
A 35 <1%
B 35 1.5%
C 30 <1%
D 36 3% (less if all branches included)
E 69 <1%
F 120-160 Unknown
G 7 2%
Exhibit B – Marketing of Service Case Amount Comments
A Moderate Web, Displays, Flyers, Lib. Instr, Newspaper
B Extensive Web, Branding, Articles, Lib. Instruction
C Moderate Target Pop., Flyers, Bookmarks, Library Instruction, Plan to Increase
D Extensive Web, Newspaper, Ads, Signs, Posters, Table Tents
E None Limited hours precluded marketing push.
F Little Pilot training & exploration period. G Extensive Email Announcement, Meetings, Giveaways,
E-Newspaper, Library Instruction
Exhibit C – Marketing of Service –Low Vol.Case Amount Comments
A Moderate Web, Displays, Flyers, LUI, Newspaper
B Extensive Web, Branding, Articles, LUI
C Moderate Target Pop., Flyers, Bookmarks, LIU, Plan to Increase
D Extensive Web, Newspaper, Ads, Signs, Posters, Table Tents
E None Limited hours precluded marketing push.
F Little Pilot training & exploration period. G Extensive Email Announcement, Meetings, Giveaways,
E-Newspaper, LIU
Exhibit D – Hours of ServiceCase Hours of Service
A M-F 3-8pm 20 hrs. per week (changed hours)
B M-Th 1-7pm, F 1-5pm 28 hrs. per week
C M-Th 5-8pm 12 hrs. per week
D M-F 1-5pm 20 hrs. per week
E Su-Th 7-10pm 18 hrs. per week
F M-Th 2-8pm 24 hrs. per week
G M-F 10am-5pm 35 hrs. per week
Exhibit E – Length of ServiceGiving VR a Fair Trial
Case Trial/Pilot Length of service
A 1/2002 – 4/2004 26 months
B 1/01 – 8/02 9/2001 – 12/2002 7 months pilot
15 months
C 5/2002 – 4/2004 21 months
D 10/2001 – 4/2002 6 months
E 1/02 - 5/02 1/2003 – 5/2004 5 months pilot
14 months
F 9/2004 – 3/2003 6 months
G 7/2002 – 2/2004 20 months
Giving VR a Fair Trial …
Shortest services ran 6 months (2 services) Longest run was 26 months Average of 17 months
Exhibit F – Service Evaluation
User
Eval?
Type of Eval?
Users
Satisfd?
Comments
A Y Survey & Trans.
Yes Mixed librarian perceptions
B Y Survey & Assmt.
Yes Low (19%) return rate.
C N ? Planned 2nd yr. survey
D N ?
E Y Survey Yes 68% “very satisfied”
40% return rate
F Y Survey Yes Staff satisfied too
G Y Survey Yes Users highly satisfied, low return rate.
Impact of Closing Services
Common Trends Demise had limited impact. Users did not notice. Email reference (phone service) more
promotion, more use. Unique Responses
Staff relieved, less pressure.
Lessons Learned
If doing a multiple case study, don’t put a number in your title for conference papers.
Consortium necessities: Full-time coordinator Variety of libraries
Some reference questions are better answered by email.
More Lessons Learned…
Critical factors: Staff training & enough volume for staff to
stay primed, comfortable staffing patterns User needs assessment
What hours for service? What software?
User evaluation Marketing Design prominent & easily identifiable link
The Mystery Remains UnsolvedOngoing Investigation… Season 2? Future Research Directions Low volume, what are critical factors? Have identified 3 additional cases.
Perhaps more will be identified today.
More rigorous examination of documents. Research on successful services – what factors
ensured their success? User expectations of service – what hours, days?
More on users. More research on non-users
Future Directions for Services
Investigate free & simpler software IM or refTracker
Consortia Share Cost Share Staff
Continue to scan landscape
There’s Hope…Future Directions for CasesCase Reinstate? Comments
A Maybe In consortium, if users want it
B Maybe Simple Chat, IM, or RefTracker, Summer ’05
C Yes If budget & staffing restored
D Maybe Possible if consortium
E Maybe Possible if consortium (doubtful)
F Yes Reorganization in process, hope to reopen in Jan. ’06
G Maybe Something free like IM
Conclusion
PPT slides will be posted at:
RUSA Reference & Statistics Committee Sitehttp://www.ala.org/RUSA
Thanks & Questions???
Marie L. Radford, Ph.D., Rutgers [email protected]. Kathleen Kern, MLS, University of Illinois at [email protected]