Amy R. Willis. The differences in perception of librarians� roles in supporting the practice of evidence-based medicine as they are represented in the literature targeted toward healthcare professionals versus the literature targeted toward librarians. A Master�s Paper for the M.S. in L.S degree. July, 2004. 55 pages. Advisor: Claudia Gollop

This paper explores what, if any, identifiable differences exist in the way health care

professionals and library professionals perceive the potential contributions of librarians to

the practice of evidence-based medicine. The question this paper seeks to answer is: Are

the actions performed by health science librarians in supporting the practice of evidence-

based medicine presented differently in the literature targeted toward healthcare

professionals than in the literature targeted toward health science librarians? Forty-five

journal articles were analyzed to determine what actions librarians were depicted as

performing in order to support evidence-based medicine. Nine of the articles were from

healthcare provider-oriented journals, and 36 of the articles were from librarian-oriented

journals. The results of this study suggest that librarians� roles in supporting the practice

of evidence-based medicine are represented as having more variety and depth in the

librarian-oriented literature when compared to the healthcare provider-oriented literature.

Headings:

Medical Librarians

Evidence-Based Medicine

Duties of Librarians

THE DIFFERENCES IN PERCEPTION OF LIBRARIANS� ROLES IN SUPPORTING THE PRACTICE OF EVIDENCE-BASED MEDICINE AS THEY ARE

REPRESENTED IN THE LITERATURE TARGETED TOWARD HEALTHCARE PROFESSIONALS VERSUS THE LITERATURE TARGETED TOWARD

LIBRARIANS

by Amy R. Willis

A Master�s paper submitted to the faculty of the School of Information and Library Science of the University of North Carolina at Chapel Hill

in partial fulfillment of the requirements for the degree of Master of Science in

Library Science.

Chapel Hill, North Carolina

July 2004

Approved by

_______________________________________ Claudia Gollop

1

Table of contents

Table of contents����������������������������..1

Introduction������������������������������..2 Review of Literature���������������������������3 Methods��������������������������������5 Results��������������������������������12 Discussion������������������������������..16 Conclusion������������������������������.18 Notes��������������������������������..19 Appendix A: Articles retrieved from PubMed search������������......21

Appendix B: Results from the health science library literature����������26

Appendix C: Results from the healthcare provider journals�����������.46

Appendix D: Phrases not used from the 45 articles included in the content analysis�..48

Bibliography�����������������������������..52

2

I. Introduction

The health care profession has witnessed and embraced the emergence of

evidence-based medicine over the past two decades. The core skills involved in the

practice of evidence-based medicine are: 1)literature retrieval, 2)critical appraisal,

3)information synthesis, 4)and the ability to act on the results. The first 3 of these skills

are also essential to librarianship, and it seems almost intuitive to a library professional

that health sciences librarians should play an essential role in supporting the practice of

evidence-based medicine. As Judith Palmer writes:

Evidence-based practice requires some expertise in information handling and a range of information skills that include the ability to frame questions, devise appropriate strategies, locate information, interrogate sources, judge relevance, understand bias, record and organize information, interpret and analyze, re-organize/re-order information and present evidence to others. These skills are commonly possessed by library and information professionals, but not necessarily by other health professionals.i While it may seem obvious to librarians that they can contribute to the practice of

EBM, health professionals are often unaware of the services that a librarian can provide

other than simply acquiring requested literature. This paper explores what, if any,

identifiable differences exist in the way health care professionals and library

professionals perceive the potential contributions of librarians to the practice of evidence

based medicine. More precisely, the question this paper seeks to answer is: Are the roles

of health science librarians in supporting the practice of evidence-based medicine

presented differently in literature targeted toward healthcare professionals than in the

literature targeted toward health science librarians?

3

II. Review of Literature The literature on librarians� role in supporting evidence-based clinical practice

focuses on a variety of settings including workshops, hospital libraries, and clinicians�

forums on diagnosis and treatment. As expected, most of the articles which delineate the

variety of librarians� activities in these settings are written by information professionals

rather than medical professionals.

Bexon and Falzon emphasized the teaching role of the librarian in a workshop

setting.ii They participated in the 2001 and 2002 �How to Teach Evidence-Based

Clinical Practice� workshops in Hamilton, Ontario, Canada. In their experience, the roles

of the librarians included teaching strategies for question formulation, devising search

strategies, and introducing workshop participants to a range of information resources

relevant to evidence-based clinical practice.

Williams and Zipperer identified the following ways in which hospital librarians

can assist nurses with their information needs: 1)performing on-demand literature

searches and document delivery, 2)providing internet and search training,

3)creating/maintaining area-specific intranet sites, and 4)locating and purchasing

reference resources to ensure the resources available at the nursing stations are current.iii

Atlas (et al.) focused on the ways in which librarians could assist medical

residents in finding evidence-based clinical information through participation in Morning

Report, a weekly forum on treatment and diagnosis. These Morning Reports were held by

the Department of Family and Community Medicine, School of Medicine at the

University of Louisville. The librarians were from the Kornhauser Health Sciences

Library, and the timeframe for the study was January through June of 2002. When the

4

librarians began to attend the ongoing forums, the plan was for the librarians to

participate in the weekly question identification and research assignments alongside the

residents. As Atlas writes:

Residents would be asked to perform their own search, print out the search, answer a question on the relevance of the retrieved citations, and answer a question on their satisfaction with the results of their search. . . . The residents� techniques and results were to be compared to those of the librarians.iv

The goals of this plan were to improve the searching skills of the medical residents, to

increase the residents� use of evidence-based decision making, to enable the librarians to

better understand the information-seeking skills needed by the residents, and to increase

awareness of the Kornhauser Health Sciences Library�s resources for teaching and aiding

health care professionals to access pertinent literature.v

However, the librarians were unable to carry out the plan as written due to several

factors, including the informal nature of the forums which resulted in the lack of a clearly

stated search question for the week, and the question generated the previous week often

failed to be readdressed. Therefore the librarians did not have on outlet during these

forums to share their search strategy and lessons they had obtained. Furthermore, when

residents did present the articles they had found, there was no discussion of their search

strategies or problems they may have encountered. However, as the months progressed,

the librarians became more comfortable jumping into the group discussions whenever

they felt their input might be valuable, and thus were able to share specific examples of

question formulation, searching behavior, and resource awareness.

As evident from the literature, multiple behaviors can be identified in the

librarian�s support of evidence-based practice. These include investigating the

information needs of the health care professionals, teaching question formulation and

5

searching skills, promoting awareness of resources, performing searches for the health

care professionals, obtaining and delivering materials, and providing access to area-

specific intranet sites.

ÍII. Methods

The assessment of whether the roles of health science librarians in supporting the

practice of evidence-based medicine are presented differently in literature targeted toward

healthcare professionals than in the literature targeted toward health science librarians

began with a literature search using MEDLINE via PubMed. It should be noted that

MEDLINE does not index all of the biomedical literature being published. Particularly,

it does not contain some of the nursing and allied health literature indexed in CINAHL.

However, it is the largest biomedical database available and thus was chosen for use in

this study.

The following search strategy was used: Evidence-Based Medicine [MeSH] AND

(Librarians [MeSh] or librarian*[text word]). No date restrictions were applied to the

search. The search was conducted on March 21, 2004 and 60 articles were retrieved.

Because the terms Evidence-Based Healthcare, Evidence-Based Practice, and Evidence-

Based Nursing are all mapped to the MeSH heading Evidence-Based Medicine, the

search strategy was effective in retrieving articles that used these related terms.

However, the search also retrieved articles on the subject of Evidence-Based

Librarianship because this term is also mapped to the MeSH heading Evidence-Based

6

Medicine; articles that focused on using evidence-based methods to practice better

librarianship rather than to practice better medicine had to be excluded from the study.

A search using alternative words for librarian was attempted. The phrase

information specialist, while not a MeSH heading, resulted in 37 hits as a quoted phrase

in a text word search. To include this term, the following search was performed:

Evidence-Based Medicine [MeSH] AND (Librarians [MeSh Terms] or librarian*[text

word] or �information specialist�[text word]). The search also resulted in 60 hits.

Combining this search with the original search also produced 60 hits, which verified that

the two result sets contained the same 60 citations. The phrase information professional

is not a MeSH heading and was not found in a text word search as a quoted phrase. It

should be noted that a text word search in PubMed is not a search of full text articles, but

rather a search of words in the title, abstract, other MeSH terms, MeSH subheadings,

chemical substance names, and personal names as subjects. The phrase information

professional was found several times in the text of the articles used in the study and was

included in the content analysis of these articles.

Fifteen out of 60 articles, all from health science library journals, had to be

excluded from the study. Fourteen of these were on the subject of evidence-based

librarianship, and one focused on how librarians could apply the methodology of

evidence-based medicine to a purchasing agency (see appendix A).

The next step was to divide the result list into the categories of

�librarian/information professional-oriented� or �health professional-oriented� type. The

journal was considered of the librarian/information professional-oriented type if it

primarily was produced for and/or by library or information professionals involved in the

7

provision of health sciences information; this set will heretofore be referred to as the

librarian-oriented journals. Likewise, the journal was considered of health-professional-

oriented type if it was primarily produced for and/or by professionals who provide patient

care; this set will heretofore be referred to as the healthcare provider-oriented journals.

As shown in Table 1, there were a total of 36 articles and eight journal titles in the set of

results from the librarian-oriented journals. There were a total of nine articles and nine

journal titles in the set of results from the healthcare provider-oriented journals.

Table 1: Journal titles and number of articles in each result set Librarian Journal Titles Number

of Articles

Healthcare Provider Journal Titles Number of Articles

*Bulletin of the Medical Library Association

9 Academic Emergency Medicine

1

Health Information and Libraries Journal 10 Academic Medicine 1 Health Libraries Review 5 British Medical Journal(BMJ) 1 International Journal of Medical Informatics

1 Central European Journal of Public Health

1

International Journal of Technology Assessment in Health Care

1

Journal of Family Practice 1

*Journal of the Medical Library Association

2 Journal of Health and Social Behavior

1

Medical Reference Services Quarterly 7 Journal of Professional Nursing 1 National Network: the Newsletter of the Hospital Libraries Section of the Medical Library Association

1 Medical Teacher

1

Nursing Economics 1 Total Articles: 36 Total Articles: 9

*Bulletin of the Medical Library Association changed its title to Journal of the Medical Library Association in 2002.

For most of the titles, it was obvious which to which category the journal title

belonged. Two of the titles were not so clear-cut: International Journal of Medical

Informatics and International Journal of Technology Assessment in Health Care.

Although neither of these publications expressly target health sciences librarians as an

audience, the areas of informatics and technology assessment seem to be more in the

8

purview of information professionals than health care providers; therefore the articles

from these journals were assigned to the librarian-oriented journals category.

The next step in the study was to code the content of the 45 articles. For the

articles available in HTML format, the web browser�s �find on this page� function was

used to find occurrences of the terms librarian, information specialist, or information

professional, or the plural forms of these terms. For the articles not available in HTML

format, each document was manually scanned twice to find the occurrences of these

terms. Only the occurrences of librarian actions in the body of the article were counted;

occurrences in abstracts, tables, acknowledgements, or appendices were not included.

The original plan for this content analysis was to only include sentences with a

clear subject-verb structure in which the librarian or information professional is the

subject who performs a specific action. However, it soon became clear that many

sentences indicated an activity performed by a librarian without following this subject-

verb formula. Additionally, there were many sentences that contained negative

references to an activity, such as a clinical librarian was not present or librarian-

mediated searches were not used. Even in the negative references, it was clear that a

means through which a librarian could support EBM was being considered. Since the

goal of this study was to examine what differences exist in the depictions of librarians�

contributions to the practice of EBM as they appear in the librarian-oriented journals

versus the healthcare provider-oriented journals, the strict subject-verb criteria was

abandoned. Any statement which contained a reference to an activity performed by a

librarian was included, even if the reference was negative. As will be discussed later,

there were some statements that had to be excluded because they were too general or

9

vague, but an effort was made to use as many statements in the study as possible. Rather

than counting the occurrence of the terms librarian, information specialist, or

information professional (or their plural forms), the actions associated with the

occurrences of these terms were counted because many sentences depicted a librarian

performing multiple actions in support of EBM.

The following 12 �action categories� were used to code the librarians� activities:

1. Provide access to information, deliver or disseminate information. This is a broad

category which includes the general management of information resources, allocating

funds to purchase information resources, and document delivery. This category also

includes general statements about librarians providing access to information in which no

precise means of accessing the information is indicated. Statements in which librarians

are depicted helping healthcare providers find information but no specific method (i.e.

searching a database, doing a reference interview) is specified are also contained in this

category.

2. Increase awareness of EBM resources, promote the concept of EBM. This is another

broad category that includes general statements in which librarians are described as

supporting or encouraging the EBM concept. Activities associated with increasing

healthcare providers� awareness of available EBM resources are also included here.

3. Teach question formulation and searching skills. This category includes the activities

associated with teaching healthcare providers how to search databases effectively.

4. Perform literature searches, mediate searches. Included here are all statements which

describe a librarian doing a literature search or using a specific resource to locate

information. The conducting of reference interviews is also included in this category

10

because the only article that specifically mentions the reference interview discusses it

only as a precursor to performing a literature search.vi

5. Evaluate resources, filter information, participate in critical appraisals. This broad

category consists of the activities associated with evaluating the literature or evidence. It

also includes general statements that describe librarians as participants in the systematic

review process, but no specific action is specified.

6. Assist in research projects, conduct research on behalf on clinicians. This category

consists of librarians adding to the pool of evidence by assisting with or conducting

medical research. Additionally, it includes statements in which the librarian goes beyond

locating and evaluating evidence and applies the evidence to a specific patient�s condition

in response to a clinician�s request, or the librarian performs additional work to verify the

conclusions found in the medical literature.

7. Develop or organize resources that support the teaching or practice of EBM.

Included here are activities such as creating topic-specific websites or Critically

Appraised Topic databases, and developing materials to support the teaching of EBM,

such as search hedges or a preparatory list of key skills clinicians need to perform

searches.

8. Teach or participate in EBM workshops, collaborate with faculty or clinicians

teaching EBM skills. Included here are statements in which librarians participate or teach

EBM workshop that are targeted toward healthcare providers, but it does not include

specific statements about teaching searching skills (these are included in category 3). It

also consists of collaboration between librarians and medical school faculty to support the

teaching of EBM.

11

9. Participate in medical faculty/student meetings such as Morning Reports or journal

clubs. This category includes any statement in which a librarian is present at routine

faculty/student meetings, such as Morning Report or a journal clubs, where the goal is to

increase the practitioners� or students� medical knowledge.

10. Increase understanding of the skills clinicians need to practice EBM, participate in

studies aimed at improving clinicians� skills. Included in this category are instances

where librarians actively seek to increase their understanding of how healthcare

providers� EBM skills can be improved. It also consists of librarians participating in

studies of how clinicians can practice EBM more effectively.

11. Improve librarians� skills, participate in studies of how librarians� skills can be

improved. This category includes librarians taking classes to improve their ability to

support EBM. It also consists of librarians conducting or participating in studies of how

to increase the EBM skills among librarians.

12. Participate in clinical services or patient round. Included in this category is any

reference to a librarian being present on patient rounds or participating in another aspect

of clinical services. It also consists of statements that mention a clinical librarian but do

not indicate a specific action.

It was expected that the occurrences of the action categories would be much more

prevalent in the librarian-oriented journals. Therefore, the content analysis was

performed separately for the two types of literature. The number of instances in which a

librarian supports the practice of EBM in some way in the collective body of each type of

literature was counted, and each instance was assigned to its appropriate action category.

Then for the two types of literature, the percentage of each action category represented

12

versus the total number of librarian actions was calculated (i.e. If a total of 200 indicating

terms are found in the librarian-oriented literature, and 10 of them fall in the Category 1,

then this category constitutes 5% of all the librarian activities). Once the percentages

were separately derived from the two types of literature, a comparison was made of the

percentages of the two categories to see if significant differences exist. A difference of 5

percentage points was considered significant. For example, if Category 1 represents 30%

of the actions in librarian library journals and 35% of the actions in healthcare provider-

oriented journals, then the difference would be considered significant. All decimals were

rounded to the nearest whole number.

ÍV. Results

A total of 633 actions were counted in the 36 articles from the librarian-oriented

journals, which averages to approximately 18 occurrences per article (see appendix B) .

There were 81 statements from the librarian-oriented journals that contained the terms

librarian, information professional, or information specialist or their plural forms but did

not reference an action included in the 12 categories. There were several reasons this

occurred. Some articles contained general statements, usually in an introduction or

conclusion, about the librarians� availability to assist healthcare providers or support

EBM but did not mention a specific means of doing so. Occasionally the action was too

specific and did not easily fit into one of the categories, but creating a new category for

such a specific action would have been excessive. An example of this is the following

statement: ��where the Librarian actively contributes to e-government by managing

13

materials coming into the Ministry of Defence, linking personnel manuals and associated

management material.�vii In other statements, the librarian was the subject of a verb and

no action could be attributed to the librarian. For example, �A curriculum was

developed�and marketed to students in the MHPE programs, clinical teaching faculty,

and librarians.�viii Additionally, instances in which �librarian� was used in a phrase that

functioned as a proper noun, such as in the title of a course (for example, �EBM and the

Medical Librarian�) were also excluded from the content analysis.

A total of 36 actions were counted in the nine articles from the healthcare

provider-oriented journals, which averages to about 5 occurrences per article (see

appendix C). There were 52 statements from the healthcare provider-oriented journals

that contained the terms librarian, information professional, or information specialist or

their plural forms but did not reference an action included in the 12 categories. This

number is disproportionately large because one article repeatedly used the term �librarian

resident� to describe medical residents whose approach to practicing EBM was

consulting any published resource to quickly solve the dilemma at hand, as opposed to

the �researcher residents� who actively evaluate and interpret the literature. It is clear

from the article that terms �librarian residents� and sometimes just �librarians� are

referring to the medical students and not information professionals.ix All of the phrases

in this article, with the exception of one occurrence in the beginning of the article which

referred to an information professional being present at morning meetings, had to be

excluded from the content analysis (see note on Article 29 in appendix D).

Results from the content analysis can be seen in Table 2.

14

Table 2: Results from Content Analysis 1 Action Category Percentage of

occurrences in Librarian-oriented journals

Percentage of occurrences in Healthcare provider-oriented journals

Difference in percentage points

1. Provide access to information, deliver or disseminate information.

5% 28% 23%

2. Increase awareness of EBM resources, promote the concept of EBM.

2% 0% 2%

3. Teach question formulation and searching skills. 12% 6% 6%4. Perform literature searches, mediate searches. 18% 25% 7%5. Evaluate resources, filter information, participate in critical appraisals.

10% 11% 1%

6. Assist in research projects, conduct research on behalf on clinicians.

2% 0% 2%

7. Develop or organize resources that support the teaching or practice of EBM.

5% 3% 1%

8. Teach or participate in EBM workshops, collaborate with faculty or clinicians teaching EBM skills.

11% 14% 3%

9. Participate in medical faculty/student meetings such as Morning Reports or journal clubs.

4% 6% 2%

10. Increase understanding of the skills clinicians need to practice EBM, participate in studies aimed at improving clinicians skills.

7% 3% 4%

11. Improve librarians� skills, participate in studies of how librarians� skills can be improved.

18% 0% 18%

12. Participate in clinical services or patient rounds.

7% 3% 4%

The action categories which had at least a difference of five percentage points were

categories 1,3,4, and 11. Because category 11, improve librarians� skills or participate in

studies of how librarians skills can be improved, represents activities heavily discussed in

librarian-oriented journals but not likely to be mentioned in healthcare provider-oriented

journals, the percentages were recalculated with the omission of category 11. These

results are displayed in Table 3.

15

Table 3: Results for Analysis 2 Action Category Percentage of

occurrences in Librarian-oriented journals

Percentage of occurrences in Healthcare provider-oriented journals

Difference in percentage points

1. Provide access to information, deliver or disseminate information.

6% 28% 22%

2. Increase awareness of EBM resources, promote the concept of EBM.

2% 0% 2%

3. Teach question formulation and searching skills. 15% 6% 9%4. Perform literature searches, mediate searches. 21% 25% 4%5. Evaluate resources, filter information, participate in critical appraisals.

12% 11% 1%

6. Assist in research projects, conduct research on behalf on clinicians.

3% 0% 3%

7. Develop or organize resources that support the teaching or practice of EBM.

6% 3% 3%

8. Teach or participate in EBM workshops, collaborate with faculty or clinicians teaching EBM skills.

14% 14% 0%

9. Participate in medical faculty/student meetings such as Morning Reports or journal clubs.

5% 6% 1%

10. Increase understanding of the skills clinicians need to practice EBM, participate in studies aimed at improving clinicians skills.

9% 3% 6%

11. Improve librarians� skills, participate in studies of how librarians� skills can be improved.

omitted omitted omitted

12. Participate in clinical services or patient rounds.

8% 3% 5%

With category 11 omitted, category 4 dropped below the significant range, categories 1

and 3 remained in the significant range, while categories 10 and 12 entered the significant

range.

The major limitation of the study is the small number of librarian actions that

appear in the healthcare-provider oriented journals (36) compared to the number of

librarian actions (633) that appear in the librarian-oriented journals. Therefore one action

in the healthcare-provider journals represents approximately 3% of the occurrences,

16

while one action in the librarian-oriented journals represents about 0.15% of the

occurrences; these differences in action to percentage ratios may skew the results.

V. Discussion

Category 1, provide access to information, contained the highest difference in

percentage points. This category accounts for 28% of the actions in healthcare provider-

oriented journals, while only accounting for 5% in Analysis 1 and 6% in Analysis 2 in the

librarian-oriented journals. This would appear to indicate that healthcare providers are

generally aware that librarians can provide information, but they may not have and in-

depth understanding of the range of information services that librarians provide.

Category 3, teach question formulation and searching skills, contained 12% of the

occurrences in Analysis 1 and 15% of the occurrences in Analysis 2 in the librarian-

oriented journals; only 6% of the occurrences were fell into this category in the

healthcare provider-oriented literature. This suggests that often healthcare providers do

not recognize that 1)improvement in searching skills are needed, or 2)librarians have the

ability to teach these skills.

Category 4, perform literature searches, represented a large percentage of

librarian activity for both types of literature: 18% (Analysis 1) and 21% Analysis 2) for

the librarian-oriented journals, and 25% for the healthcare provider-oriented journals.

The difference in percentage points was significant (7%) in Analysis 1 and non-

significant (4%) in Analysis 2. These relatively large percentages indicates that both

17

librarians and healthcare providers view mediated searching as a prominent and valuable

service that librarians provide in support of EBM.

Category 10, increase understanding of the skills clinicians need to practice

EBM/participate in studies aimed at improving clinicians� skills, contained 7% of the

occurrences in Analysis 1 and 9% of the occurrences in Analysis 2 in the librarian-

oriented journals; only 3% of the occurrences were fell into this category in the

healthcare provider-oriented literature. The difference in percentage points was non-

significant (4%) in Analysis 1 and significant (6%) in Analysis 2. This implies a lack of

awareness among healthcare providers of the efforts librarians make to address the EBM

skill deficits among medical professionals.

Category 11, improve librarians� skills/participate in studies of how librarians�

skill can be improved, contained 18% of occurrences in Analysis 1 and, as noted earlier,

was omitted in Analysis 2; 0% of the actions fell into this category in the healthcare-

provider oriented literature. The fact that this action category did not appear in the

healthcare provider-oriented literature but is heavily represented in the librarian-oriented

literature can be attributed to the vocations of intended audiences of these journals. It

certainly indicates that librarians are putting forth much effort to empower information

professionals to increase their involvement in the EBM process.

Category 12, participate in clinical services or patient rounds, contained 7% of

the occurrences in Analysis 1 and 8% of the occurrences in Analysis 2 in the librarian-

oriented journals; 3% of the occurrences in the healthcare provider-oriented literature fell

into this category. The difference in percentage points was non-significant (4%) in

Analysis 1 and significant (5%) in Analysis 2. This may point to the lack of prevalence

18

of clinical librarianship and the difficulty in funding such programs, or that many

healthcare providers have never considered the possibility of a librarian participating in

clinical services.

VI. Conclusion

The results of this study show that librarians� roles in supporting the practice of

evidence-based medicine are represented as having more variety and depth in the

librarian-oriented journals when compared with the healthcare-provider journals. The

study indicates that healthcare providers understand the librarian�s role in terms of

providing general access to EBM resources, conducting literature searches, and

participating in EBM workshops and EBM curriculum development; however, they are

less aware of the librarians� potential to teach searching skills and participate in clinical

services.

The goal of this study was for information professionals to better understand and

thus address the discrepancy between their perception of what they can do and health

professional�s perception of the medical librarian�s potential services. Knowledge of this

difference in perceptions should enable the librarians to better market themselves and

their services. A recent issue of Medicine on the Net asked the following question in the

title of its lead article: �Open access: Will it spell the end of the medical library?� The

gist of the article is that with the advent of open access publishers who make scientific

research available free on the Internet as opposed to charging high subscription fees,

hospital managers may question the need to maintain and staff physical libraries.x Of

19

course, all libraries, not just medical libraries, have to combat the assumption that

Internet access decreases the necessity of libraries. What is unique to the medical library

is the flourishing of evidence-based medicine, a practice which requires the literature-

searching and evaluation skills at which librarians are supposed to excel. If health

professionals� perceptions of librarians� capabilities in supporting evidence-based

practice can be broadened, hopefully the librarians will be better able to make the

necessity of their positions obvious to a manager who is making budget cuts.

The future of evidence-based practice and how healthcare practitioners will

respond to it is uncertain. However, it is clear from the proliferation of articles on

evidence-based medicine in the librarian literature that information professionals stand

ready and willing to teach and assist those healthcare providers in improving their EBM

skills.

Notes

i Palmer, Judith, �Health and Higher Education--a Migrant�s Tale� SCONUL Newsletter 20

(2000): 19-23.

iiBexon, Nicola and Louise Falzon, �Personal Reflections on the Role of Librarians in the

Teaching of Evidence-Based Healthcare� Health Information and Libraries Journal 20 (2003): 112.

iiiLinda Williams and Lorri Zipperer, �Improving Access to Information: Librarians and Nurses

Team Up for Patient Safety� Nursing Economics 21.4 (2003): 200.

ivMichel C. Atlas, et al., �Case Studies from Morning Report: Librarians� Role in Helping

Residents Find Evidence-Based Clinical Information� Medical Reference Services Quarterly 22.3 (2003):

5.

vAtlas 4.

20

vi Andrew Booth, Alan J. O�Rourke, and Nigel J. Ford, �Structuring the Pre-Search Reference

Interview: A Useful Technique for Handling Clinical Questions� Bulletin of the Medical Library

Association 88.3 (2000): 239-246.

7 Caroline Plaice and Pam Kitch, �Embedding Knowledge Management for NHS South-West:

Pragmatic First Steps for a Practical Concept� Health Information and Libraries Journal, 20.2 (2003): 77.

viii Josephine L. Dorsh, Susan Jacobson and Carol S. Scherrer, �Teaching EBM Teachers: a Team

Approach� Medical Reference Services Quarterly, 22.2 (2003): 109.

ix Stefan Timmermans and Alison Angell, �Evidence-Based Medicine, Clinical Uncertainty, and

Learning to Doctor� Journal of Health and Social Behavior 42.4 (2001): 342-59.

x Catherine Zandonella, �Open Access: Will It Spell the End of the Medical Library?� Medicine

on the Net 9.11 (2003): 1-7.

21

Appendix A: Articles retrieved from PubMed search

**Indicates that article was excluded from study. 1. Davidson RA, Duerson M, Romrell L, Pauly R, Watson RT. Evaluating evidence-based medicine skills during a performance-based examination. Acad Med. 2004 Mar;79(3):272-5. 2. Lappas E. Exploiting research information: comments about the different databases in health care (DIMDI, OVID CD-ROM). Cent Eur J Public Health. 2003 Sep;11(3):155-9. 3. Jacobs SK, Rosenfeld P, Haber J. Information literacy as the foundation for evidence-based practice in graduate nursing education: a curriculum-integrated approach. J Prof Nurs. 2003 Sep-Oct;19(5):320-8. 4. Williams L, Zipperer L. Improving access to information: librarians and nurses team up for patient safety. Nurs Econ. 2003 Jul-Aug;21(4):199-201. 5. Bexon N, Falzon L. Personal reflections on the role of librarians in the teaching of evidence-based healthcare. Health Info Libr J. 2003 Jun;20(2):112-5. 6. Deshpande N, Publicover M, Gee H, Khan KS. Incorporating the views of obstetric clinicians in implementing evidence-supported labour and delivery suite ward rounds: a case study. Health Info Libr J. 2003 Jun;20(2):86-94. 7. Plaice C, Kitch P. Embedding knowledge management in the NHS south-west: pragmatic first steps for a practical concept. Health Info Libr J. 2003 Jun;20(2):75-85. 8. Beverley CA, Booth A, Bath PA. The role of the information specialist in the systematic review process: a health information case study. Health Info Libr J. 2003 Jun;20(2):65-74. **9. Booth A. Using research in practice. Health Info Libr J. 2003 Jun;20(2):116-8. **10. Booth A, Eldredge JD. 'and even evidence-based librarianship'? Health Info Libr J. 2003 Jun;20 Suppl 1:1-2. **11. Eldredge JD. The randomised controlled trial design: unrecognized opportunities for health sciences librarianship. Health Info Libr J. 2003 Jun;20 Suppl 1:34-44. **12. Koufogiannakis D, Crumley E. Facilitating evidence-based librarianship: a Canadian experience. Health Info Libr J. 2003 Jun;20 Suppl 1:73-5. **13. Doney L, Stanton W. Facilitating evidence-based librarianship: a UK experience. Health Info Libr J. 2003 Jun;20 Suppl 1:76-8.

22

14. Atlas MC, Smigielski EM, Wulff JL, Coleman MT. Case studies from morning report: librarians' role in helping residents find evidence-based clinical information. Med Ref Serv Q. 2003 Fall;22(3):1-14. 15. Brown JF, Nelson JL. Integration of information literacy into a revised medical school curriculum. Med Ref Serv Q. 2003 Fall;22(3):63-74. 16. Dorsch JL, Jacobson S, Scherrer CS. Teaching EBM teachers: a team approach. Med Ref Serv Q. 2003 Summer;22(2):107-14. 17. Tomasulo P. PD(X)MD: a new clinical information tool for patient care. Med Ref Serv Q. 2003 Summer;22(2):77-83. 18. Rigby E, Reid L, Schipperheijn JA, Weston L, Ikkos G. Clinical librarians: a journey through a clinical question. Health Info Libr J. 2002 Sep;19(3):158-60. 19. Grandage KK, Slawson DC, Shaughnessy AF. When less is more: a practical approach to searching for evidence-based answers. J Med Libr Assoc. 2002 Jul;90(3):298-304. 20. Crites GE, McDonald SD, Markert RJ. Teaching EBM facilitation using small groups. Med Teach. 2002 Jul;24(4):442-4. **21. Crumley E, Koufogiannakis D. Developing evidence-based librarianship: practical steps for implementation. Health Info Libr J. 2002 Jun;19(2):61-70. **22. Eldredge JD. Evidence-based librarianship: what might we expect in the years ahead? Health Info Libr J. 2002 Jun;19(2):71-7. **23. Booth A. Evidence-based librarianship: one small step. Health Info Libr J. 2002 Jun;19(2):116-9. 24. Wildridge V, Bell L. How CLIP became ECLIPSE: a mnemonic to assist in searching for health policy/management information. Health Info Libr J. 2002 Jun;19(2):113-5. 25. Bradley DR, Rana GK, Martin PW, Schumacher RE. Real-time, evidence-based medicine instruction: a randomized controlled trial in a neonatal intensive care unit. J Med Libr Assoc. 2002 Apr;90(2):194-201. 26. Ely JW, Osheroff JA, Ebell MH, Chambliss ML, Vinson DC, Stevermer JJ, Pifer EA. Obstacles to answering doctors' questions about patient care with evidence: qualitative study. BMJ. 2002 Mar 23;324(7339):710.

23

27. Reid L, Ikkos G, Hopkins W. Clinical librarians at Barnet Primary Care NHS Trust: addressing the information requirements of clinical governance. Health Info Libr J. 2002 Mar;19(1):52-5. **28. Booth A. From EBM to EBL: two steps forward or one step back? Med Ref Serv Q. 2002 Fall;21(3):51-64. 29. Timmermans S, Angell A. Evidence-based medicine, clinical uncertainty, and learning to doctor. J Health Soc Behav. 2001 Dec;42(4):342-59. 30. Alper BS, Stevermer JJ, White DS, Ewigman BG. Answering family physicians' clinical questions using electronic medical databases. J Fam Pract. 2001 Nov;50(11):960-5. 31. Helmer D, Savoie I, Green C, Kazanjian A. Evidence-based practice: extending the search to find material for the systematic review. Bull Med Libr Assoc. 2001 Oct;89(4):346-52. 32. Booth A, Brice A. Increasingly the health information professional's role in supporting evidence-based practice requires familiarity with critical appraisal skills, resources and techniques. Health Info Libr J. 2001 Sep;18(3):175-7. 33. Weigand JV, Gerson LW. Preventive care in the emergency department: should emergency departments institute a falls prevention program for elder patients? A systematic review. Acad Emerg Med. 2001 Aug;8(8):823-6. 34. Falzon L, Booth A. REALISE-ing their potential?: Implementing local library projects to support evidence-based health care. Health Info Libr J. 2001 Jun;18(2):65-74. 35. Booth-Clibborn N, Milne R, Oliver S. Searching for high-quality evidence to prepare patient information. Health Info Libr J. 2001 Jun;18(2):75-82. **36. Booth A. Turning research priorities into answerable questions. Health Info Libr J. 2001 Jun;18(2):130-2. **37. Booth A. Will health librarians and related information workers ever work together to create an international network? Health Info Libr J. 2001 Mar;18(1):60-3. **38. Eldredge JD. Evidence-based librarianship: an overview. Bull Med Libr Assoc. 2000 Oct;88(4):289-302. 39. Booth A, O'Rourke AJ, Ford NJ. Structuring the pre-search reference interview: a useful technique for handling clinical questions. Bull Med Libr Assoc. 2000 Jul;88(3):239-46.

24

40. Rader T, Gagnon AJ. Expediting the transfer of evidence into practice: building clinical partnerships. Bull Med Libr Assoc. 2000 Jul;88(3):247-50. 41. Garrison JA, Schardt C, Kochi JK. Web-based distance continuing education: a new way of thinking for students and instructors. Bull Med Libr Assoc. 2000 Jul;88(3):211-7. 42. Wolf D. Evidence-based complementary care: the librarian's role on a Complementary Medicine Assessment Committee. Natl Netw. 2000 Jul;25(1):12-3. 43. Sable JH, Carlin BG, Andrews JE, Sievert MC. Creating local bibliographic databases: new tools for evidence-based health care. Bull Med Libr Assoc. 2000 Apr;88(2):139-44. **44. Eldredge JD. Evidence-based librarianship: searching for the needed EBL evidence. Med Ref Serv Q. 2000 Fall;19(3):1-18. 45. Holtum EA. Librarians, clinicians, evidence-based medicine, and the division of labor. Bull Med Libr Assoc. 1999 Oct;87(4):404-7. 46. McGowan JJ, Winstead-Fry P. Problem Knowledge Couplers: reengineering evidence-based medicine through interdisciplinary development, decision support, and research. Bull Med Libr Assoc. 1999 Oct;87(4):462-70. 47. Scherrer CS, Dorsch JL. The evolving role of the librarian in evidence-based medicine. Bull Med Libr Assoc. 1999 Jul;87(3):322-8. **48. Farmer J, Williams D. Research into practice? Health Libr Rev. 1999 Jun;16(2):137-40. 49. Pyne T, Newman K, Leigh S, Cowling A, Rounce K. Meeting the information needs of clinicians for the practice of evidence-based healthcare. Health Libr Rev. 1999 Mar;16(1):3-14. 50. Topfer LA, Parada A, Menon D, Noorani H, Perras C, Serra-Prat M. Comparison of literature searches on quality and costs for health technology assessment using the MEDLINE and EMBASE databases. Int J Technol Assess Health Care. 1999 Spring;15(2):297-303. 51. Cavanaugh BB, Horne AS. Ovid's evidence-based medicine reviews: a review of a clinical information product. Med Ref Serv Q. 1999 Fall;18(3):1-14. 52. Gallagher PE. Getting started in evidence-based health care: a guide to resources. Med Ref Serv Q. 1999 Winter;18(4):1-10.

25

53. Cumbers BJ, Donald A. Using biomedical databases in everyday clinical practice: the Front-Line Evidence-Based Medicine project in North Thames. Health Libr Rev. 1998 Dec;15(4):255-65. 54. Rose S. Challenges and strategies in getting evidence-based practice into primary health care--what role the information professional? Health Libr Rev. 1998 Sep;15(3):165-74. 55. Hicks A, Booth A, Sawers C. Becoming ADEPT (Applying Diagnosis, Etiology, Prognosis, and Therapy Programme): delivering distance learning on evidence-based medicine for librarians. Health Libr Rev. 1998 Sep;15(3):175-84. 56. McKibbon KA. Evidence-based practice. Bull Med Libr Assoc. 1998 Jul;86(3):396-401. 57. Glanville J, Smith I. Evaluating the options for developing databases to support research-based medicine at the NHS Centre for Reviews and Dissemination. Int J Med Inf. 1997 Nov;47(1-2):83-6. 58. Haines M. Libraries and the R&D strategy: a way forward. Health Libr Rev. 1996 Dec;13(4):193-201. 59. McCarthy LH. Evidence-based medicine: an opportunity for health sciences librarians. Med Ref Serv Q. 1996 Winter;15(4):63-71. **60. Booth A, Hey S. From intelligence to evidence-based healthcare: a purchasing odyssey. Health Libr Rev. 1995 Dec;12(4):249-59.

26

Appendix B: Results from the health science library literature

Type of action Phrase as it occurred it the article Article number

role of the librarian . . . included assisting the participant to find relevant articles

5

variety of resources available via the librarian 6 information professionals contribute towards those pillars through providing and managing the information content

7

this �know-how� about the organization and the provision of access to, and training in, the knowledge base, provides opportunities for information professionals to influence the very heart of the organization in which they are employed

7

the former health authority resource collection and health promotion resources, be managed by an information professional

7

Health librarians have traditionally focused on ...providing access to a range of information tools

8

two contexts for the involvement of information specialists in evidence-based health care have been identified. The first role, �evidence consumer�,�.may include involvement in locating and disseminating evidence

8

highlighted the valuable contribution librarians can make to systematic reviews, ...predominantly...reference management and document delivery

8

others have argued that information professionals are not naturally good disseminators

8

librarians have the skills required to become advocates for evidence-based health care ... by making resources... readily available online

8

librarian brought the books 14

librarians requested an internet-connected laptop . . . [so that} residents could do searches

14

medical librarians�are now focusing on providing more effective methods of information retrieval, usually through electronic means

19

librarians are encouraging their users to build their own personalized portals to library home pages for quick access to resources

19

librarians are also looking to handheld computers to provide this information in a clinical setting

19

the heart of the clinical librarian service is the provision of patient-focused information to clinicians

27

librarians have their own �evidence cart� [comprised of] a laptop loaded with The Cochrane Library, EMBASE Psychiatry�

27

the clinical librarians are able to provide these services on a more detailed basis and can also be pro-active in providing relevant information when it becomes available

27

after meetings with participating librarians it became clear that it would be more effective for each site to use the money available to tailor resources to meet their differing local needs

34

technological wonders that librarians continue to embrace�create new avenues for accessing information

45

efficient and effective access to this evidence�has traditionally fallen under the purview of librarians

46

partnerships between librarians and clinicians in information acquisition in support of health care delivery

46

providing software support [for the databases] through the librarians 53 local librarians were asked to take delivery of the peripheral hardware to be the contact for the installation

53

Provide access to information, deliver or disseminate information

librarians�might consider reviewing their document delivery systems and procedures, with the aim of reducing delivery time

53

27

pressure on the librarian to provide in-house document delivery, interlibrary loans, and full text services faster and more efficiently

56

needs of librarians were considered in greater detail since these information brokers were seen to be key in the dissemination and use of the NHS CRD databases

57

target librarians more effectively [in dissemination strategies] 58 librarians looking for new and innovative ways in which to improve physician access to important and relevant clinical information

59

Total 29 role of the librarian�included�introducing the group to a range of information resources

5

librarians have the skills required to become advocates for evidence-based health care� by providing current awareness or digest services to clinicians

8

librarians were able to use the funding opportunity to demonstrate that their role in identifying, using and marketing resources to match need is pivotal to the whole process of EBHC

34

the committee was made up of�a librarian�.The purpose of the committee was to promote the notion of basing clinical practice on evidence

40

promotional activities are carried out by the clinician or the librarian 40 training, librarian support and database accessibility appear to have been the most significant factors [in the firms using databases successfully

43

role of the librarian in helping all those in primary health realize the opportunities presented by the movement towards evidence-based practice

54

STLIS, in common with many other library services and individual librarians, has for some years been encouraging the practice and implementation of evidence-based health care

55

librarians has �stripped out� the main points and take home messages [from the EBM workshop] and circulated these to Clinical Audit and Public Health staff within the hospital

55

a national database of systematic reviews was identified as the top priority by librarians in a recent survey prioritising improvements to the quality of knowledge of healthcare

57

librarians do offer current awareness services 58 librarians to use this knowledge of their clients to bring relevant RDD publications to their attention

58

Increase awareness of EBM resources, promote the concept of EBM

Total 12 librarians have managed to . . .share and teach their knowledge of searching and accessing evidence-based information resources

5

librarians participated fully in the critical appraisal group sessions, but our particular role was to teach searching skills to participants using evidenced-based electronic resources

5

role of the librarian...included ...enabling the participants to recognize the role and importance of question formulation

5

role of the librarian...included ...prompting the participants to formulate a clinical question and devise an appropriate search strategy

5

librarian...to lead a discussion on how the article under investigation might have been located

5

librarian taught a 2-hour searching skills session 5 librarians teach a single searching session 5 participants...make appointments with their librarian to receive extra searching tuition

5

having a librarian 'on tap' had been a distinct advantage, as searching issues could be discussed for each specific article

5

Teach question formulation and searching skills

as librarians we often teach [searching] skills at a certain level 5

28

health librarians have also increasingly assumed an educator role by teaching health professionals and/or students how to search for and to appraise evidence themselves

8

librarians are the ideal resource personnel to assist residents in developing the skills of formulating questions . . . and performing the searches

14

the thought was that librarians would be better able to perform this role [of assisting in developing the skills of formulating questions] if they were present

14

librarian was able to review a number of points about searching MEDLINE

14

librarian further demonstrated the availability of second tier criteria 14 librarians would post examples of their searches 14 librarians will give feedback to the residents so that eventually residents will meet all criteria for searching

14

librarians discovered that the topics from Morning Report were often discussed at noon on Wednesdays at rounds. It was suggested that one Wednesday noon session each month be devoted to librarians and residents searching together so that each resident would have the opportunity to learn

14

librarians teach a hands-on information skills workshop 15 librarians�provide feedback to the students including recommendations on methods for conducting a more efficient search

15

meet with a librarian to review searching techniques 15 librarians teach a hands-on workshop on [searching] evidence-based medicine resources

15

librarian lead the group in finding information related to questions that arose during the patient visits

15

the ICM directors and the librarians decided to discontinue the [finding evidence] exercise

15

librarians realized that the faculty also needed instruction on using information resources effectively

15

librarian from the GMR was invited to teach the PubMed session 16 librarian again presented the session on searching PubMed 16 the clinical librarians use their contextual knowledge to guide the formation of clinical questions

18

the gathering of such information to help in conducting well-formed searches is not unique to the work of the clinical librarian

18

librarians are now teaching targeted searching techniques 19 librarians can emphasize the location of relevant retrieval�by training clinicians to search

19

approached librarians at Taubman about teaching a hands-on session for faculty on searching for evidence-based literature

25

residents�worked one-to-one with a librarian and learned EBM searching techniques in MEDLINE

25

assess the potential for improving residents� EBM searching skills in MEDLINE through real-time librarian instruction

25

minimum of several training sessions with librarian contact would be required

25

the desire to keep each encounter [teaching searching skills] between NICU residents and librarians to a brief duration

25

whether real-time instruction and feedback by medical librarians resulted in improved EBM searching

25

librarian provided active instruction based on the nature of the question and the searcher�s level of skill

25

librarians most frequently provided instruction on use of MeSH, use of evidence-based search strategies

25

does real time librarian instruction in a dynamic clinical environment improve residents� EBM searching skills

25

librarian instruction resulted in the improvement of searching skills 25

29

librarians working with residents found that interruptions during searching sessions were common

25

librarian would assist with training and improving search techniques 27 librarians were particularly aware of the opportunities to target previously disadvantaged groups [for training]

34

availability of the new computer equipment acted as a catalyst for increased librarian involvement in end-user training

34

librarians may be involved in training clinicians in evidence-based practice searching techniques

40

individual coaching sessions may be led by the�librarian, depending on the need for searching expertise across clinical areas

40

increases the clinicians� computer and searching skills and leads to more relevant interaction with the librarian

40

training of end users in searching is a role that many librarians in teaching hospitals have adopted

40

providing clinicians with basic searching skills�improves the ability of the clinician to work collaboratively with librarians

40

librarians�have sometimes promoted�the concept of end user searching

45

librarians have advanced the notion that all clinicians can search effectively

45

one or two instructional [searching] sessions with a librarian 45 librarians also need to reconsider the amount of their limited time they should spend teaching their clientele how to search

45

librarians began teaching end-user searching 47 residents assigned to search with the librarian 47 as they become familiar with the librarians, residents have been observed to be more likely to seek help in literature searching

47

collaborative searches occur weekly, but librarians must be flexible because the residents� time is tightly scheduled

47

librarians at LHS sites currently teach a class that introduces students to EBM and the search skills necessary to access information

47

interviews with librarians from participating Trusts as well as regional librarians�were undertaken to discuss their experience of training clinicians in searching and information retrieval skills

49

need the assistance of a librarians to use electronic databases 49 rely heavily on librarians to cascade skills in information searches 49 I wouldn�t really try to brief someone else in how to carry out a search, we have an excellent librarian for that

49

questions librarians are most often asked by clinical searchers 49 trainers recruited librarians to train this [searching skills] sessions 53 discussing the search problems with the librarian 53 felt the need for closer liaison with a librarian [to aid in searching] 53 three respondents already worked closely with their librarian [in searching]

53

librarians experience difficulties in training clinicians to use effective search techniques

53

impact of the librarian�s intervention in teaching residents to perform effective literature searches

54

librarians also need to learn new skills in their increasing role as teachers and trainers to help clinicians identify citations and ways to find them

56

librarians have�experience in training others in searching the literature

58

librarians should�provide�training in information retrieval techniques

58

librarians are available to help residents form their questions into searchable terms and phrases

59

30

physicians, medical students, and other health care professionals can benefit from programs offered by librarians to improve their ability to formulate accurate search strategies

59

Total 75 this feeling [of clinicians not having time to search for evidence] persisted in spite of the assurance of help from the clinical librarian in conducting the search

6

these are the areas in which information professionals have their core skills: indexing, organization and retrieval of information

7

health librarians have traditionally focused on their expert database searcher role in identifying evidence for clinicians

8

highlighted the valuable contribution librarians can make to systematic reviews, ... predominantly, literature searching

8

this[the role of literature searcher] is perhaps the most familiar role for information specialists

8

searches done by . . . medical librarians 14 librarian searchers have always been considered to be the ultimate standard

14

clinicians and librarians achieved comparable recall 14 librarians achieved significantly better precision 14 librarians were significantly better at both [recall and precision] 14 librarians would search for an answer 14 librarians who had sought and discovered information 14 she [the librarian] went searching 14 librarian performed the following search 14 librarian�s initial strategy was to perform a MEDLINE search 14 librarians bring�a knowledge of literature and skills in information searching and retrieval

16

It [the resource] tells the doctor and the librarian what signs and symptoms are connected with medical conditions, what to do for those conditions based on the current and quality-filtered evidence in the literature

17

providing fairly in-depth background information for those difficult search questions that librarians sometimes struggle with

17

this resource could also be used for cut-to-the chase answers during clinical librarian rounds, conferences, and morning reports

17

the clinical librarian will either search for the information there or will take the search away and report back

18

librarian was able to�carry out a comprehensive search for information

18

this [mediated search] was not an isolated example of the impact of the clinical librarians

18

the clinical librarians have carried out over 1000 searches for information related to patient care

18

the clinical librarians bring a depth of contextual information to the searches they perform

18

MEDLINE has been the database of choice for�librarians seeking medical information

19

this mnemonic is now commonly used by health information professionals when searching for best current evidence

24

health librarians are also increasingly being called on to provide management-related bibliographic references which also require well-planned search strategies

24

use of the mnemonic CLIP as a search aid evolved from the use of certain questions which we, as experienced health policy librarians, found we automatically asked in a reference interview

24

librarians conducted extensive literature searches 25

Perform literature searches, mediate searches

both librarians noted their lack of medical training as an obstacle to formulating an answer [to doctors� questions about patient care]

26

31

librarians have, since the project began, responded to a wide variety of questions from team members

27

asked if the clinical librarian could provide some evidence that the client could take to the GP

27

role of the clinical librarian in these groups is to search for evidence and clinical guidelines

27

number of searches carried out by the existing library services and the clinical librarians has more than doubled since the project began

27

the searches carried out by the clinical librarians 27 searches carried out by the clinical librarians 27 researchers and librarians search the subject-specific and specialized databases

31

they [the search strategies] were developed by librarians with extensive knowledge and understanding of the databases

31

because clinicians..rely more on evidence found in the literature base�, librarians play a key role in the advancement of EBP

34

role librarians have in supporting health care practitioners to find the evidence they need

34

with the increasing availability of computerized bibliographic information systems, the �pre-search reference interview� became an important stratagem for targeting what was perceived initially as an expensive but powerful addition to the librarian�s armory

39

the online computer reference service has changed both the setting for the reference interview and the relationship between librarian and client

39

librarians have been taught to apply more scientific methods to dealing with user�s information needs

39

questions librarians asked during the reference interview 39 use of a structured anatomy to elicit a more complete picture of the originating information need�may improve communication and move the librarian toward a more active role in the delivery of patient-specific, evidence-based health information

39

search might be conducted by an intermediary, typically a medical librarian

39

goals of the medical librarian in supporting clinical practice through intermediary searching

39

clinician and librarian to conduct a �real time� search 39 whether structuring the literature search request would �prove acceptable to participating librarians

39

librarians were able to translate inquiries of clinical origin into three- or four-part questions

39

in this process [of finding evidence],�librarians�can expedite the transfer of evidence into practice

40

librarian/clinician partnership has been fostered�with an ulterior goal of facilitating the transfer of evidence into practice

40

librarians should be responsible for wholly performing the literature searches

40

librarians are an integral part of the evidence gathering 40 ongoing partnerships between clinicians and librarians can improve communication, enabling librarians to better answer clinical questions

40

identified by the librarian using a detailed search strategy 40 literature searching support from the librarian 40 role of librarians as searchers 40 librarian skills such on online searching and information management are showcased

40

literature search provided by the librarian 42 librarian was appointed to the committee [to support its efforts with comprehensive searches]

42

librarian begins with MEDLINE searching 42

librarian also searches general literature databases 42

32

effective literature searches still usually require the expertise of information professionals

43

it makes more sense for librarians, rather than clinicians, to perform literature searches

45

can the same not be said of the expertise and experience [in performing literature searches] that librarians bring to the health care enterprise

45

librarians are in danger of neglecting those clinicians who would prefer to have their searches mediated by librarians

45

librarian could execute the entire search 45 can librarians possibly find enough time to perform mediated searching 45 more librarians are needed [to do mediated searching] if evidence-based medicine is to become a reality

45

hospital librarian, who then selects the most suitable databases, formulates and performs the search

45

instead of abandoning the notion of mediated searching, librarians should be looking for ways to make it easier

45

librarians can conduct mediated searches 45 librarians�develop complex search strategies 46 librarians�are able to negotiate an information problem, to formulate and run complex searches for needed output

46

this tool needs to combine the best precepts of evidence-based medicine--as gleaned from an extensive knowledge net, preferable supported by highly skilled medical librarians

46

librarian has the expertise to access the body of knowledge 47 librarian gathers, or points the user to, bibliographic citations or sources

47

comments regarding the [searching] services provided by the librarians and other information managers were on the whole extremely positive

49

I have asked the librarian to get articles for me [because I don�t know how to carry out a search]

49

availability of librarians or other facilitators who would assist in conducting the actual literature searches

49

librarians with the ability to conduct searches for evidence 49 librarians at each agency ran literature searches 50 allows�librarians access to literature in support of �evidence-based medicine

51

medical librarians, whose expertise in searching 52 assist the librarian in designing search strategies 52 information professionals are potentially an important group in its implementation because few clinicians�have to time or the skills to locate relevant research findings

53

librarians could encourage an exchange of ideas on the best ways of using databases to find evidence

53

any changes in librarians� searching techniques 53 availability of librarian-mediated searches 53 clinicians in the project did not request librarians� help with searches 53 quality of end-user searching compared with that by librarians 53 experienced end-users equaling librarians [in retrieval skills] 53 librarians retain a significant advantage in specificity [in retrieval skills]

53

effectiveness of librarian-mediated searches 53 librarian-mediated searches were not used 53 lack of confidence in the librarians� ability [to perform mediated searching]

53

librarian-mediated search service was not publicized 53 librarian-mediated search service was inconvenient to use 53 librarians� cost-effectiveness as searches could be increased 53 increased usage by clinicians of librarians� skills, particularly in database searching

53

librarians should be encouraged to offer mediated searches 53

33

utilizing the skills of librarians for effective retrieval 53 literature searches were performed by expert librarian searchers 54 this is the step [finding journal literature] that is most important for librarians

56

because clinicians who practice EBP rely more on evidence found in the literature base�, librarians play a key role in the advancement of EBP

56

librarians need to develop and keep their search skills strong 56 users were librarians with greater skills in information retrieval 57 librarians have skills in information retrieval, organization, and management

58

these searches, however, were conducted by expert librarian-searchers 59 searches in this study were also conducted by expert librarian-searchers

59

access to expert librarian-searchers 59

Total 112 participants asked their librarian to join in the appraisal session 5 allowing librarian tutors to participate in the critical appraisal sessions 5 it is important for information specialists to become members of an interdisciplinary systematic review team

8

two contexts for the involvement of information specialists in evidence-based health care have been identified�.The second role, �evidence producer�, involves contribution to the production of systematic reviews

8

the role of the information specialist has therefore evolved�to being quality filterers, critical appraisers

8

information professionals have assumed a more proactive role, filtering the results�, as well as, in some cases, formally appraising the quality of research papers

8

ScHARR had worked with Anne Brice, formerly of the University of Oxford, to develop a series of CRItical Skills Training in Appraisal for Librarians (CRISTAL) checklists

8

although critical appraisal checklists, such as CRISTAL, do now exist for librarians, these require further refinement

8

information specialist is now regarded as a partner in the research process�for health technology assessment(HTA) reviews

8

information professionals involved in the National Institute for Clinical Evidence(NICE)technology appraisals are named as report authors

8

aim of this paper is to discuss the role of the librarian in the systematic review process

8

the project [of a systematic review]was undertaken by a small team composed entirely of information professionals

8

the review team consisted entirely of information professionals 8 ten roles that the information professional can have in the systematic review process

8

information specialist�s role in critical appraisal is less well accepted 8 it is unusual, although not unheard of , for this procedure[data extraction] to be undertaken by information professionals

8

for this review, important data were elicited from the included studies by two information professionals. A sample(10%) was double-checked by a verifier, who was also an information specialist.

8

no reports have been identified to date that describe the information specialist�s role in synthesizing the results for systematic reviews. In the review presented here, all three information professionals were involved in this task.

8

information specialists do possess the full range of skills required to undertake systematic reviews in a health information setting

8

Evaluate resources, filter information, participate in critical appraisals

ten different roles for the information specialist in the systematic review process have been identified.

8

34

information professionals should be afforded the opportunity to become more involved in systematic review activities

8

exact involvement of the information specialist will ultimately depend on the individual skills and experience of both the information specialists and the other members of the review team

8

authors are not proposing that systematic reviews of health care topics should be undertaken solely by information specialists

8

information specialists will not have the necessary medical expertise when working on health care topics [for systematic reviews]

8

information professionals working on the review�had previously attended an intensive systematic review training course

8

information specialists possess many of the characteristics required to undertake systematic reviews

8

specific sites are selected by librarians and then reviewed by the faculty to ensure appropriate content

15

librarian volunteered to evaluate the searches 15 librarians are increasingly being asked to provide information that is filtered by scientific rigor

19

the evaluation of the validity of medical information�by librarians 19 librarians�are in a wonderful position to study the strengths and weaknesses of these new tools to determine whether they are truly evidence based and present patient-oriented evidence that matters

19

librarians can play a significant role in helping clinicians evaluate the clinical value of these resources

19

librarians are participating in systematic reviews 31 health information professional�s role in supporting evidence-based practice requires familiarity with critical appraisal skills

32

suggest three particular barriers to greater participation by librarians in critical appraisal

32

not only do many clinicians consider that librarians are ill-equipped to assume an extended role in exploiting research literature, but librarians themselves may have concerns about taking on this mantle

32

would librarians be more amenable to take the lead in critical appraisals

32

health librarians to apply critical appraisal skills in their day-to-day practice

32

workshops have introduced consumer health information specialists to appraising evidence

35

information specialists appraise the information in advance 35 review of resources has traditionally been a function of librarians 40 the [selection and review of] content falls under the purview of the clinicians and the librarians

46

librarians are�continually reviewing journal and monographic literature and suggesting modifications and additions to extant couplers

46

veteran librarians felt uncomfortable with the new roles they were being asked to assume: �quality filtering of literature

47

librarians recognize and select articles as effectively as physicians 47 librarians used article title, abstract, and journals title as reasons for selection

47

librarians focusing more on medical subject headings [to select appropriate articles]

47

hospital librarians have also been concerned with the issue of quality filtering

47

citing future trends for hospital librarians, Klein noted that value-added service roles, such as information filtering, should grow

47

librarians should read the full text of the most pertinent articles�, identify and extract the information relevant to the clinician at hand

47

Nagle also called for libraries and librarians to change significantly�by stating, �Emphasis is�on obtaining the best information available for a given situation,�and to winnow out the quality from the quantity of available information

47

35

EBM require the librarian to�identify, select, evaluate, and synthesize literature

47

EBM offers librarians the opportunity to participate fully in the information process [by evaluating the evidence they find]

47

librarians became more proficient in�filtering the literature 47 library to �move from library services to decision support services� 47 librarians are able to do this [highlight clinically relevant research papers of sound methodology] to clinicians� satisfaction

53

librarians felt confident in critical appraisal skills 53 potential role of the information professional as a critical appraiser 54 enhanced role for librarians�to include the delivery of critical appraisals

55

librarians should identify, obtain, and sift evidence from printed publications, on-line databases

58

librarians as �quality filters� 58

Total 61 not only are information specialists appropriately placed to oversee the running of the project, but many also possess necessary project management skills.

8

information specialists are expected to take a more proactive role in managing the references associated with large research projects

8

this is truly a case of the information professional evolving from the �research provider� to �research doer�

8

to double-check the conclusions reached, the clinical librarian contacted the local drug information service

18

having carried out a comprehensive search of evidence, the clinical librarian was able to report back to the psychiatrist that the patient�s symptoms�

18

membership [of the committee established to evaluate alternative medicines] has expanded to include�a medical librarian

42

research team was lead by a librarian and a Ph.D. nurse researcher 46 librarian and one of the nurse researchers did a detailed analysis of each patient�s chart and selected couplers against which the findings could be run

46

collaborative research between librarians and clinicians in the form of two pilot studies

46

there are few published works that list librarians among the authorship groups

46

librarians are not the only health care professionals routinely excluded from research protocols

46

importance of collaborative research because it can expand the range of research librarians can undertake

54

librarians must be encouraged�to add to the evidence by conducting research relevant to the health information problems and needs in their own institutions

58

Assist in research projects, conduct research on behalf on clinicians

Total 13 recognition of the role of librarians as organisers of information coming into the Trusts

7

information professionals working on KM initiatives�[including] the Library and Information Services Manager manages the new post of web development officer and takes responsibility for the editorial content of the Trust�s Intranet

7

information professionals taking the lead in developing their organizations� websites

7

information professionals are taking a lead in dissemination activities. Examples include the development of resources such as the National electronic Library for Health(NeLH) and CAT (Critically Appraised Topics) banks.

8

Develop or organize resources that support the teaching or practice of EBM

librarians will develop a set of key skills [to teach clinicians] for doing a search

14

36

librarians can place [EBM resources] into the proper level of the searching hierarchy

19

Librarians are uniquely aware of the intricacies of a broad range of search systems, allowing them to rank their usefulness more easily

19

librarians� organization of searching hierarchies 19 residents received EBM search hedges, developed by the librarians at the University of Rochester Medical Center Library

25

the committee [which includes librarians] therefore developed a comprehensive Website�that uses the strength and expertise of both the librarian and the clinicians

40

librarian has developed inclusion criteria with which clinicians can review potential links to sources

40

once the librarian identifies possible sites [for inclusion in the EBM guide], they are transferred to a database for further review by clinicians

40

clinicians have collaborated with the librarian in writing detailed instructions for searching each resource

40

hospital librarians can work collaboratively with all members of clinical areas by using the Internet and the Web to manage information for clinicians

40

collaboration of the librarian and the clinicians encompasses more than the creation�of the guide

40

librarian member of the committee must be knowledgeable about other Web guides [to assist in creating one]

40

librarian�s role is�to support the educational component of the committee�s work and to develop library resources in complementary medicine

42

make it possible for medical librarians to create easy-to-use, locally maintained bibliographic databases

43

librarians can use local databases to compile the results of monthly SDI searches and to present the search results to clinicians through a Web browser interface

43

method for librarians to create Web-searchable bibliographic databases was developed

43

allows librarians to compile the results of monthly SDI searches into a single Web-searchable database

43

librarians need to be actively engaged in the development and refinement of end user information tools

45

Problem Knowledge Couplers are created and refined through the work of�clinicians, informaticians, and librarians

46

Problem Knowledge Couplers are the result of an interdisciplinary collaboration among clinicians, informaticians, and librarians

46

they [Problem Knowledge Couplers] are the product of a collaboration among clinicians, informaticians, and librarians

46

couplers, build on the work of librarians, enable access to literature for clinical problem solving when it is impractical for a librarian to be present

46

can librarians create easily accessible knowledgebases of best evidence that will expedite the transfer of evidence into practice

47

Web pages that evaluate EBP information were compiled by health information professionals

52

librarians have developed search �hedges� that are especially useful in quality-filtering literature searches

52

develop librarians� abilities in critical appraisal and in filtering search results

53

described the librarian�s role [as a trainer in searching] in an evidence-based medical programme

54

librarians have produced excellent summaries of EBP for the Internet 56

Total 32 Teach or participate in EBM workshops,

as a pilot scheme initiated by Canadian health care librarians, a librarian tutor was also assigned to each small group

5

37

librarian tutors [participating in the workshop] originated from the USA, Canada and the UK

5

librarians were chosen for their knowledge of EBM and critical appraisal, and their searching and teaching skills

5

due to the success of librarian involvement in 2001 5 workshop organisers invited librarians to participate 5 librarian's role in the group sessions varied 5 librarian tutor also worked individually with participants in their group 5 feedback from the small groups about librarian involvement was positive

5

librarians also benefited from the workshop 5 workshops utilize librarians in different ways 5 dedicated librarian tutor such as the McMaster experience 5 librarian as a workshop participant 5 her expertise as an information specialist was highly valued by her fellow participants during the group sessions

5

dedicated librarian integration into the McMaster workshop 5 librarian spends all week with their small group 5 participants benefit from receiving one-to-one tuition from their librarian

5

they [the participants] could also utilize the skills and knowledge of the librarian when needed

5

participants using this facility could receive teaching support from the librarians [during the workshop]

5

librarians and participants [in the workshop] made suggestions for inclusion on this website

5

[the workshop] gave the librarians a chance to experiment with their own teaching skills

5

workshops where librarians attend full-time 5 integrating librarians into these workshops 5 it involves the participation of many librarians for a whole week 5 several �floating� librarians for groups to share 5 it is important...that librarians give as much to the process as the tutors 5

librarians should attend these courses 5 it was felt by librarians and participants alike, that ideally all rooms should have a networked PC

5

disadvantage for the librarian is that this was a very labour-intensive model

5

during our regular librarian tutor meetings , it did become apparent that a few librarians had problems with the group tutors being hostile or not allowing them sufficient time to teach

5

groups do not have dedicated librarian tutors 5 do not have access to librarians during the rest of the workshop 5 having a dedicated librarian could be seen as a luxury 5 this is not the best utilization of the librarian�s skills 5 [librarians attending the workshops] will provide an opportunity for the benefits and cost-effectiveness of the different models of librarian involvement to be evaluated

5

librarians�had long been interested in being involved in the curriculum and in teaching information literacy to health science students

15

librarians serve on curricular committees 15 Keck School of Medicine initiated a curriculum revision process that provided the librarians with the opportunity to put in place some of the recommended medical infomatics objectives

15

process provided the librarians of the Norris Medical Library with the opportunity to provide input and make changes at a curricular level

15

in EBM workshops, collaborate with faculty or clinicians teaching EBM skills

librarians participated in the curriculum revision process 15

38

librarians gathered information about the intended curriculum revision process and generated ideas as to the library�s involvement

15

librarians decided that core principles�and integrated cases�provided the best opportunity for library involvement

15

librarian be included on the Curriculum Revision Committee 15 librarians have been members of the various medical school curriculum committees

15

Keck School already viewed librarians as a part of the education process

15

committee chairs welcomed the librarians to their committees and were receptive to their ideas

15

librarian�wrote a proposal for an information literacy curriculum 15 librarian�was asked to chair a sub-committee 15 librarian�provided information support to the committee 15 library volunteered one of its librarians to be a co-chair of the committee partnering with a clinician as the other co-chair

15

librarian co-chair wrote a draft report that was eventually adopted by the committee

15

greater appreciation among the faculty of the potential for the contribution of librarians to the medical school and has re-emphasized the role of librarians as partners in the educational process

15

librarians continue to serve as committee members 15 faculty and librarians are enthusiastic about the integration of informatics into the curriculum

15

librarians have made modifications to the information resources workshops and to the literature search projects

15

can partnerships between librarians and clinicians lead to effective educational programs

16

approach that teams information and health professionals as both teachers and learners

16

librarians�have been active in curriculum-integrated instruction and liaison programs

16

librarians are essential to the EBM process and key to establishing an institutional base to support the teaching and practice of EBM

16

EBM is a collaborative effort of health and information professionals, and these two groups should partner in the teaching and practice of EBM

16

working in teams of health and information professionals in class activities reinforces the contributions and skills that each group brings to the EBM process

16

happy with the mix of health and information professionals [in the class]

16

due to�the wish to develop more librarians as teachers of the EBM process

16

partnerships between librarians and clinicians lead to effective EBM instructional programs

16

clinical librarians are now planning to offer a workshop on basic critical appraisal skills

27

identify the areas where librarians could participate in training and supporting health care professionals

41

librarians at all four sites�have long been involved in curriculum-integrated instruction

47

veteran librarians felt uncomfortable with the new roles they were being asked to assume: teaching EBM skills

47

integrate librarians into the College of Medicine EBM activities and curricular changes

47

as practitioners and teachers of EBM, librarians establish a collaborative relationship with health care professionals

47

brings librarians into the hospitals and classrooms, into graduate and undergraduate training programs, and into curriculum and decision-making capacities

59

39

more reports of librarian-developed education and training programs 59 Total 71

librarians were approached about participating in Morning Report 14 librarians attend the weekly Morning Report 14 sessions attended by the librarians 14 comfort levels of the individual librarians affected interaction with the medical staff [during Morning Report]

librarian . . . began to engage in the group discussion 14 librarian felt comfortable enough to enter the dialogue 14 librarian input into addressing questions that arose during morning report

14

librarian . . . told the Morning Report group that she had searched for �zebras�

14

librarians . . . have been participating in Morning Report 14 role of librarians in Morning Report has improved and evolved over time

14

librarians attended ICM faculty meetings 15 librarian also attended weekly resident meetings to discuss their searches

25

librarian involvement in EBM journal clubs and morning report 47 librarians are playing an active role in the COM�s EBM Special Interest Group

47

librarians� participation in morning report 47 the first year of [librarian participation in morning report] has been a series of trial and error with librarians, residents, and COM faculty struggling to find a role for the librarians that is clearly defined

47

veteran librarians felt uncomfortable with the new roles they were being asked to assume: �participating in journal clubs and morning report

47

librarians assigned in teams of two (one for backup), attend morning report one day a weeks

47

librarians are formally introduced [at morning report] 47 librarian quickly becomes a working member of the team [at morning report]

47

attendance at morning report exposes the librarians to medical terminology

47

the journal club�is another venue in which librarians seek to contribute

47

librarians� roles [in the journal club] are in evolution as they become familiar with the format and the issues addressed

47

librarian will ultimately become part of the preparation session [for the journal club]

47

Participate in medical faculty/student meetings such as Morning Reports or journal clubs

Total 24 allowed the librarians to discover how clinicians search for information 5 librarians sought to understand the information-seeking skills needed for providing EB care

14

librarians wanted to answer the following questions: What search techniques do residents use?

14

residents� techniques and results were to be compared to those of the librarians

14

the same analysis would be conducted for the librarians� searches 14 comparison would then be made between the residents and the librarians

14

an assessment would be made to identify any changes in the librarians� perceptions of the kinds of questions they may be addressing and the librarians� understanding of the residents� abilities

14

librarians did not feel comfortable asking the residents to explain their search methods

14

Increase understanding of the skills clinicians need to practice EBM, participate in studies aimed at improving clinicians� skills

librarians conducted a survey on information-seeking behavior 15

40

only the librarians knew to which study group individual residents were assigned.

25

a librarian arrived in the NICU two to three days per week, immediately following rounds [to carry out the study]

25

the subjects were curious about the presence of librarians in the NICU 25 librarians sought to identify RCTs of end-user MEDLINE training, studies in teaching EBM searching skills, studies identifying the most problematic aspects of MEDLINE searching for medical students

25

librarians evaluated all strategies with a locally designed search strategy analysis tool

25

residents�emailed the retrieval strategy and strategy to the librarians for analysis as a baseline of their searching skills

25

the study began with two librarians alternating days, while a third librarian observed to gain experience

25

librarian sat with each resident�, while the resident attempted to find clinically relevant information in MEDLINE

25

librarian completed a data collection form to capture information on the search experience

25

librarian noted whether or not it [the search question] contained the four elements of a good question

25

for those in control group, the librarian observed the search 25 librarians evaluated the quality of each search strategy 25 librarians judged the success of each search 25 librarians observed a total of thirty-five searches during the intervention phase of the study

25

librarians scored search strategies from the individual pre-, post-, and six-months post-intervention searches

25

from the perspective of the librarian investigators, the most significant outcome measure was the analysis of search strategies

25

librarians quantified and thus systematically compared subject�s search strategies

25

librarians believed that the real-time nature of this study positively affected the searching confidence and retention of EBM searching skills in the test group

25

librarians found the precision and recall of Cochrane articles to be less significant measures of successful searching skills

25

coding was conducted independently by one librarian and by one researcher

31

librarians get greater insight into the needs of clinicians 40 advantages for librarians in this partnership include a greater understanding of clinical questions and resources that can answer those questions

40

by working closely with clinicians can librarians truly identify clinician�s ongoing information needs

40

clinicians and librarians are going to have to come to a better understanding not only of the capabilities and limitations of databases but of the skills of the humans who search them

45

seeing residents challenged to make diagnosis and treatment decisions�increases librarians understanding of the environment in which physicians work

47

librarians pointed out that it is not only the technical skills to manipulate databases that clinicians lack, but also an awareness of searching strategies

49

one of the Trust librarians put this difference [in searching abilities] down to the type of type of training the various professions have had

49

librarian at one of our sites�notes the following spoken and unspoken questions which reveal the distance between mechanistic and effective use of medical databases

49

[the clinician competency] framework is derived from a combination of information sources including�opinions from librarians and information specialists

49

41

initially, there were no librarians or other specialists in information retrieval involved in the project, but it soon became clear that in-depth knowledge of the databases used in the project would be crucial

53

organize librarian support for the project 53 involvement of Trust librarians in the project 53 involve the Trust librarians as major participants in the project 53 nine librarians came to the introductory meeting 53 librarians on those [project] sites were urged to be as pro-active as possible in ensuring the firms would not be disadvantaged

53

review was by questionnaires, one for clinicians and one for librarians 53 librarians� initial concerns was that the project would generate a large number of interlibrary loan requests

53

total of 198 questionnaries were distributed�to: 65 librarians 57

Total 47 allowed the librarians to improve their critical appraisal skills and understanding of statistical concepts

5

as UK librarians, we had to quickly familiarize ourselves with resources such as Up-To-Date and StatRef

5

networking with other librarian tutors was helpful for gaining additional knowledge on available resources and searching techniques

5

the librarians sought to . . .assist the Kornhauser Health Sciences librarians in improving their ability to teach health care professionals skills necessary to access the professional literature

14

librarians reviewed work done by other medical libraries 15 professional development program for librarians 16 four-week series[of the professional development program] for LHS librarians

16

regional librarians were invited, and the course was approved for MLA continuing education credit

16

�Understanding Study Design and Statistical Concepts in Clinical Research� gave librarians a foundation for filtering search results

16

course is oriented toward the teaching career needs of both health care and health information professional

16

information professionals requested that more time be spent on statistical concepts

16

more time�on statistical analysis for the librarians 16 librarians in the group were familiar with PubMed searching, the presentation provided an update on PubMed developments

16

calls to separate librarians from health care professionals for more in-depth coverage of statistical concepts

16

librarians had an in-depth session on statistical concepts 16 librarians have become active partners in the EBM process by expanding their skills to include a better understanding of clinical research and its resulting literature

19

feedback from the librarians attending attending these [Evidence-Based Librarianship] events

24

it capitalizes on librarians� knowledge of the context of their work to introduce a rudimentary knowledge of research design and to present necessary statistics

32

it would be more realistic, both in terms of availability of the literature and librarians� familiarity with it, to focus on other forms of evidence [for the project]

32

tool for use studies was tested briefly and opportunistically in a session with librarians

32

librarians need to keep abreast of organizational and political changes and develop skills in critical appraisal, information retrieval and teaching end users to search

34

Improve librarians� skills, participate in studies of how librarians� skills can be improved

some projects aimed at�developing the professional competencies of the information professionals

34

42

librarians were enthusiastic about active involvement in the development of the project

34

librarians brought their proposals to a first project meeting in June 1997 and reported their progess at a second in October

34

interview of �key informants� comprising the librarian(s) involved at each site

34

interviews aimed to elicit �reflective practice� from the study participants, particularly the librarians

34

brief questionnaires circulated to target groups via librarians 34 local librarians acquired new skills 34 continuing professional development or training of the librarians 34 in terms of the participating librarians themselves, involvement in the project led to opportunistic, rather than planned, professional development

34

reported greater confidence in their librarians abilities and an overall increase in their skills

34

identified by some participating librarians in the design and analysis of questionnaires

34

librarians involved in future projects access the following list to maximize their likelihood of success

34

librarians involved had seized their opportunity, using it in a myriad of ways to enhance not only their service but also their own role within their organizations

34

involvement of librarians in local implementation strategies� is already leading to a plethora of similar or related projects

34

willingness of participating librarians to share their experiences 34 it would be impossible to have the same information professionals contemporaneously offer EBM-structured and minimally structured variants of the form [as part of the study]

39

the research team approached twenty participating librarians at the six sites with a questionnaire

39

if librarians responsible for implementing them found them unusable 39 librarians felt that users had reacted negatively to the EBM-structure forms

39

librarians felt that the minimally structured form were easier to use 39 two librarians in favor of the EBM-structured forms 39 twelve of the librarians felt that the EBM-structured forms provided more information about the user�s needs

39

general impression of the forms was elicited from the librarians 39 to implement this at a service level required that librarians be trained to filter clinical questions

39

librarians found the EBM-structured forms useful for clinical questions 39 librarians felt that the form was repetitive 39 preference by librarians for the minimally structured forms 39 examine more closely the training needs of librarians 39 develop indicators that will help librarians to �triage� requests according to whether they may fit appropriately within a clinical question paradigm

39

librarians must identify ways to bring relevant information to clinicians in a timely and organized fashion

40

provide health information professionals with quality, cost-effective alternatives to traditional continuing-education resources

41

objective was to provide introductory information about evidence-based medicine and roles medical librarians could undertake in educating health professionals

41

teach librarians the concept and process of practicing evidence-based medicine

41

exercise was intended to simulate how a librarian or clinician would respond when encountering a real clinical question

41

43

instructors looked for librarians [to be beta testers] with varying degrees of subject knowledge

41

librarians who were interested in learning more on the topic were recruited

41

six librarians were identified [to participate in the study] 41 three librarians [who participated in the study] were from academic settings

41

librarians have �a responsibility to continue the evolution of our education�and to commit to lifelong learning��

47

librarians will need to engage in a concerted effort to accept new roles and acquire new skills

47

the need for continuing education as first step for positioning librarians for expanded roles in an EBM environment

47

design a professional development program for librarians 47 search skills of both novice�and veteran librarians �needed enhancement to meet the responsibility of finding the best clinical evidence

47

librarians need to develop and keep their search skills strong 47 librarians need to learn new skills 47 program to provide librarians with a basic foundation in these skills 47 increase librarians� skill levels 47 EBM round table [to familiarize librarians with EBM] was established to which all LHS librarians, affiliated hospital librarians, and several representatives form the COM were invited

47

format [of the class] provided opportunity for discussion�preparing librarians for their pending participation in these arenas

47

classes were open to librarians from nearby institutions 47 fifteen librarians participated in this course 47 EBM is a process of lifelong learning�for librarians 47 even librarians not engaged in public service felt they benefited [from the course] because they would be able to apply the study design and statistical concepts to their own research

47

librarians are also encouraged to attend EBM training courses offered at EBM and Cochrane centers

47

how graduate schools might redesign curricula to prepare librarians for future work in EBM settings

47

institute a professional development program for librarians 47 librarians were encouraged to attend and/or plan the class 47 librarians chose to participate [in the class] 47 this effort [to promote librarians� professional development] �was developed and attended by UIC librarians

47

[the class] was also open to librarians working in nearby institutions 47 librarians were able to devote work time both to prepare and attend class

47

addressed the issue of deficiencies in the background of the librarians, especially in the area of statistics

47

librarians were able to address this area of knowledge, without being overwhelmed, with the help of COM faculty

47

weekly meetings [to continue the librarians� professional development] held with COM faculty and those librarians most heavily involved with the EBM process

47

message conveyed to resident librarians is that continuing education is always to be a part of their professional life

47

hearing unfamiliar medical terminology forces librarians to learn new vocabulary to understand clinical discussions better

47

librarians are reinforcing and expanding their own understanding of EBM concepts introduced in the formal statistics class

47

pharmacists, administrators, and librarians are potential audiences for the course

47

44

continuing education interventions have helped librarians develop the ability to contribute to EBM activities at UIC

47

lifelong, self-directed learning for librarians 47 positioned librarians to continue to evolve as the profession faces the 21st century

47

librarian who is beginning to learn about the intricacies of an evidence-based practice

52

an introductory text for librarians [about EBM] 52 book [about EBM] written specifically for the librarian 52 book�is designed to acquaint the librarian with terminology and theoretical basis of evidence-based searching

52

support the teaching and practice of EBM through the education of physicians and librarians

52

training for the librarians was arranged 53 variations in the training�of librarians 53 training of librarians in support of evidence-based practice 53 librarians and other information professionals may identify unmet training and education needs themselves, which will need to be addressed

54

librarians required to master new tools such as Cochrane Library and Best Evidence databases

55

delivering continuing education to NHS librarians 55 a pioneering attempt to train librarians to support evidence-based healthcare

55

training programme was set in place to equip librarians for new and extended roles

55

an orientation workshop�was attended by 30 librarians 55 intensive training to equip their librarians with the skills required for evidence-based practice

55

librarians are reluctant to spend much time away from their libraries, or traveling to training venues

55

a course for academic health librarians�using the Internet as a learning tool and learning medium

55

course gave me a real hunger to be part of a learning community of health librarians

55

a similar [distance learning] programme�for Trent librarians 55 its [the distance learning programme�s] impact to date is heavily indebted to the commitment of the South Thames librarians

55

librarians need to learn about the kinds of studies and trials that comprise the clinical research

56

librarians also need to develop skills in understanding how clinical research is done

56

value of library and librarian development programs 58 librarians must receive appropriate training in critical appraisal and research methodologies

58

Total 116 this approach can be incorporated into a clinical ward round with the support of a clinical librarian

6

clinicians and librarians are not used to working together on the 'shop floor'

6

challenges relating to training, experience and support for the clinical librarian have been explored by other authors

6

librarian services have been developed to support clinical practice 6 ways to involve librarians effectively on the delivery suite ward round 6 clinical librarian and internet access already available on the labour and delivery suite

6

librarian attended one ward round per week 6 introduction of the clinical librarian on ward rounds was a new concept for many clinicians

6

Participate in clinical services or patient round

librarian accompanying the round 6

45

librarians went on rounds with small groups of students and their faculty mentor

15

the clinical librarian project�has been operating since January 2000 18 the clinical librarians are integrated into specific multi-disciplinary teams with whom they meet at venues where information needs arise

18

the presence of the clinical librarian at the case discussions at which questions are generated allows a greater understanding of the context of the question

18

it is at this point [during the team discussions] that the team has found the presence of the clinical librarian useful

18

the number of information requests made by teams with access to the clinical librarians is far higher than by other teams

18

the provision of the clinical librarian service enables clinicians to apply the evidence to their clinical practice

18

invited librarians to attend rounds in the neonatal intensive care unit 25 the project was launched in January 2000 with 1.6 clinical librarians 27 clinical librarians are attached to specific multidisciplinary teams 27 each of the teams has its own dynamic and has chosen to use the services of the clinical librarians in different ways

27

the clinical librarians attend ward rounds 27 �control� teams have also been asked to participate�so that any development in information usage in study teams can be compared with teams not in contact with a clinical librarian

27

it is supporting clinicians [in evidence-based practice] which makes up most of the work of the clinical librarian

27

the clinical librarians are available in the clinical setting 27 the bulk of the work of the clinical librarian is aimed at facilitating evidence-based practice with respect to the care of individual patients

27

clinical librarians are meeting a previously unmet need 27 clinical librarians contribute to a number of groups involved in implementing policy to ensure that developments are evidence-based

27

this [contributing to groups that implement policies] is seen as a natural extension of the role of the clinical librarian as it again ensures that patients are in receipt of best quality health care

27

a second tenant of clinical governance is continuing professional development and this too is catered for by the clinical librarians

27

clinical librarians offer an effective way of combating some of the information needs of clinical governance.

27

clinical medical librarian programs are excellent examples of teamwork to improve patient care

46

not every hospital has a clinical medical librarian program 46 it would be virtually impossible to assign every clinician�a librarian who would be on call

46

human intervention in the form a clinical medical librarian on call twenty-four hours a day, five days a week, is impossible

46

clinical medical librarianship�was one method initiated by a librarian to meet clinical information needs

47

librarian attended rounds to identify information needs 47 CML [clinical medical librarianship] services were offered�to establish the medical librarian�s role as a valid member of the health care team

47

librarians misunderstood questions during rounds 47 librarians be well-versed in medical terminology to understand conversations on rounds thoroughly

47

presence of a librarian either on ward rounds or at clinical discussion meetings

53

librarian would need to be at ease in clinical situations and have the confidence of the clinicians

53

Total 41

46

Appendix C: Results from the healthcare provider journals

Type of action Phrase as it occurred it the article Article number

key to improving access to evidence from biomedical literature is the medical reference librarian

4

librarians can play a role . . .by managing resource purchasing [for nursing stations]

4

medical librarian can play a substantial part in acquiring and desseminating this information

4

nurse administrators can establish partnerships with a medical librarian to help staff . . . through improved access to the evidence

4

this role [of providing access to biomedical literature] is filled by a medical librarian

4

request from a medical librarian some information related to a current clinical case

4

librarian�s assistance [in finding information] 4 hospital librarians assist nurses in their information needs 4 information gained with the assistance of the medical librarian 4 librarians can provide point-to-point dissemination of information 4

Provide access to information, deliver or disseminate information

Total 10 Increase awareness of EBM resources, promote the concept of EBM

Total

0

librarian shows strategies for finding statistical data 3 expertise in refining the search question and . . . evaluating relevant results rests with the medical librarian

4 Teach question formulation and searching skills

Total 2 an expert retrieval librarian 2 librarians play an important role in evidence-based medicine because of the importance of identifying and retrieving appropriate literature

2

any expert system�provides evidence information on a specific subject, only by using the services of an expert reference librarian

2

an expert librarian�, using firstly order logic and secondly any experience in strategy could lead to better access to reference work

2

medical librarians are experts at refining the search question 4 medical librarians found clear answers for 46% of 60 randomly selected questions

30

medical librarians searched for an average of 43 minutes per questions from family physicians

30

medical librarians used MEDLINE and textbooks to answer 86 questions from family physicians

30

range of knowledge resources that librarians use 30

Perform literature searches, mediate searches

Total 9 as a medical librarian�, I would like to express my observations regarding the criteria towards the database selected

2

librarian�s ranking of criteria toward the databases 2 librarians understand the need to evaluate sources 4 selection team consisting of 3 family physicians and a medical librarian determined whether the nominated databases met our inclusion criteria

30

Evaluate resources, filter information, participate in critical appraisals

Total 4 Assist in research projects, conduct research on behalf on clinicians

Total

0

Librarian has developed a course Web site 3

47

Develop or organize resources that support the teaching or practice of EBM

Librarian has developed a course Web site 3

Develop or organize resources that support the teaching or practice of EBM

Total

1

nursing faculty and librarians began to integrate information literacy competencies into undergraduate nursing programs

3

collaboration between librarians and nursing faculty to integrate information literacy into the master�s program

3

faculty member worked with the librarian to determine what aspects of the individual course requirements were best suited to a session on information literacy

3

the 11 participants in the Pre-course�included�four medical librarians 20 medical librarians had variable backgrounds as EBM teachers 20

Teach or participate in EBM workshops, collaborate with faculty or clinicians teaching EBM skills

Total 5 an attending physician proposed having a librarian-researcher armed with an on-line computer present at morning meetings to provide EBM on the spot

29 Participate in medical faculty/student meetings such as Morning Reports or journal clubs

Total

1

clinical librarian who rated the students� searching skills 1 the searches were evaluated by a clinical librarian 1 to help validate the taxonomy [of obstacles encountered when attempting to answer doctors� questions with evidence], we first asked four volunteers (two medical librarians and two university family doctors) to answer four additional questions from the same dataset

26

Increase understanding of the skills clinicians need to practice EBM, participate in studies aimed at improving clinicians� skills Total 3

Improve librarians� skills, participate in studies of how librarians� skills can be improved

Total

0

occasional participation by the librarian in patient rounds can engender trust in the relationship between the librarian and the clinical team

4

Total 1

48

Appendix D: Phrases not used from the 45 articles included in the content analysis Article number

Phrase

2 historically, reference librarians have accepted the classification of reference sources by format orientation session . . . introduces the Health Sciences Librarian as a personal contact 3 Web site . . . allows asynchronous interaction with the librarian via e-mail during off hours nurses and patients both benefit from a partnership between nurses and librarians nurse administrator is a key person in establishing the partnership between the nursing staff and the medical librarian

4

medical librarians have a specialized skill set to complement those of practitioners a network 'designed to facilitate links between health care professionals, teachers, managers, librarians, specialists, researchers and other professionals working in the NHS family of organizations'

the Librarian actively contributes to e-government by managing material coming into the Ministry of Defence, linking personnel manuals and associated management material Trust Librarian chairs the Trust Intranet Working Group

7

we have learnt that information professionals have the capacity to understand and demonstrate the importance of a �whole systems approach� and some of the ways in which knowledge can be harnessed

8 there is an increasing boyd of literature describing the role of the information specialist in supporting the process of evidence-based health care

14 aid the residents in determining how and when to use a medical librarian a curriculum was developed�and marketed to students in the MHPE programs, clinical teaching faculty, and librarians

16

in an attempt to interest more librarians in the health information field, the local GSLIS was added as a co-sponsor of the course

17 an interesting comparison would be with�UpToDate, which has been notoriously unfriendly to libraries and librarians

18 some of these searches would have been carried out even without the involvement of the clinical librarians 19 this approach to the medical literature is similar to the tertiary-secondary-primary literature pyramid used

by information specialists 24 we welcome feedback on its [the pneumonic for searching] usefulness in order to make certain that it meets

the needs of fellow health information professionals. Article 29 uses the term �librarian resident� to describe medical residents whose approach to practicing EBM was consulting any published resource to quickly solve the dilemma at hand, as opposed to the �researcher residents� who actively evaluate and interpret the literature. It is clear from the article that �librarian residents� and sometimes just � librarians� are referring to the medical students and not information professionals. Eleven residents relied on the evidence as librarians, and six others utilized the literature more as researchers. Librarian residents expanded the source material that qualifies for EBM. librarian residents relied largely on �cheat books,� textbooks, guidelines, and review articles the next line of defense for librarians were the thick general textbooks the most sophisticated literature sources consulted by librarians consisted of on-line protocols librarians skimmed the methodology and focused on the conclusion and findings librarians consulted review articles, textbooks, and guidelines because they provided quick answers to clinical questions library residents frequently asked an attending for advice before approaching the literature library residents reserved literature searches for rare cases librarians found much evidence via the database �MD Consult� the core of EBM for librarian residents was the pragmatic reliance on literature to quickly solve the dilemma at hand librarians thought that medicine had always been steeped in EBM in contrast to the residents who used the literature as a librarian librarians highlighted the practice of checking the literature when diagnostic or treatment problems arose librarians and researcher residents expressed similar doubts about their ability to effectively evaluate a primary research article. different approaches of �librarian� and �researcher� residents

29

librarians act mor along the lines set out by Atkinson, Katzz, and Light

49

for librarians, practicing EBM with guidelines and review articles provided some comfort within the chaos of clinical training librarian residents tended to search prepackaged EBM librarians mentioned that guidelines provided and additional kind of legal comfort librarian residents remarked that primary studies inevitably generated contradictions and confusions for librarians the primary literature perpetuated confusion and led to an avoidance of such studies librarians and researchers were again at odds about the merits of evidence for librarians, any literature search was fundamentally incomplete librarians reacted against the widespread perception that EBM would take the place of clinical judgement librarian residents pointed out the EBM only touched on the �real work� of managing patients for librarians, the ultimate litmus test of reducing uncertainty and gaining certitude is having done things repeatedly in the past the philosophy lead librarian residents to avoid doing too many literature searches librarians considered literature searches, at best, a dubious tool for reducing clinical uncertainty librarians look to the research for ready answers librarians� instrumental use of the literature is more likely to confirm a dogmatic clinical practice librarians might overstate the certainty of research and take recommendations at face value the attitude of librarians for whom the urge to dominate clinical uncertainty with ready-made knowledge prevails neither librarians nor researchers expressed an abandonment of �humanitarian� values due to EBM librarians� tendency toward dogmatism with regard to literature findings librarians generally rejected a technological or scientific mindset regardless of whether they used the literature as researchers or librarians librarians will check ready-made evidence librarian residents as a group tend to prefer the hands-on handling of patients librarians still need to acquire evidence to back up their practice

our distinction of �librarian� and �researcher� residents 30 we solicited nominations for potentially suitable databases from multiple E-mail lists (including�medical

librarians) and through web searches 32

librarian participation in Evidence-Based Health Care is vital to its successful adoption at a local level librarians are in an ideal situation to become stronger partners in the improvement of health care librarians were known to be keen to support EBHC but were limited by lack of resources librarians� level of expertise [in using IT resources] differed significantly to those envisaged at the conception of the project librarians, at times, felt isolated due to a lack of individual support frustrations reported by the librarians included an expectation of support for evaluation that had not been realized librarian involvement in evidence-based health care in general

34

the skills and resources already developed by librarians in response to the demand for evidence-based health care factors in the interaction between librarian and clinical staff make an already complicated communication process even more complex information exchanges between a librarian and hospital staff revealed a high prevalence of technical jargon the ability of librarians to use highly technical language to communicate with health care professionals might cause the latter to view librarians as member of the medical team a positive reaction from the librarians to be prerequisite, in terms of both implementation and compliance, to obtaining user views if the librarians could not see a benefit from the change in procedure, it was unlikely that their users would seek to instigate such a change

39

randomization by library, rather than requestor or librarian, would require a much larger sample than study logistics allowed interest in evidence-based practice among clinicians has prompted librarians to examine their roles in all aspects of the process the important roles librarians and clinicians play in the diminishing these barriers [to evidence based practice] this type of partnership places the librarian in a position of greater visibility in the hospital demand for librarians and their skills could increase as clinical partnerships are formed

40

librarians make valuable contributions to the knowledgebase of clinical staff

50

knowledge acquired by both clinicians and librarians will increase the sophistication of dialogue between the two groups health information professionals have come to recognize that change is a constant force in today�s working environment

41

authors developed a Web-based course entitled �Evidence-Based Medicine (EBM) and the Medical Librarian� appointment of a librarian to the committee is an indication of the value of the librarian�s efforts 42 the librarian�s role is expected to change with the activities of the committee

43 of the many factors that have contributed to the Internet information explosion, two are particularly relevant to medical librarians: 1)the simplicity of using Web browsers and 2) the ease with which databases can be connected to Web browser front ends I have had the privilege of working closely with hospital librarians throughout the state of Iowa the willingness of librarians to embrace new and more efficient ways of performing their services librarians should be among the first health care professionals to adopt new technologies librarians maintain a healthy real-world skepticism when it comes to wholesale adoption of new information service models thoughts I have expressed below were formed as a result of rather candid conversations with hospital librarians

45

by basing health science library services on what clinicians, computers, and librarians do best, librarians can create within the hospital setting a knowledge-based information system the role of the librarian in this endeavor [of EBM] is not new, but rather a continually evolving process initiative taken by the librarians at the Library of Health Sciences of the University of Illinois at Chicago demonstrate one way this evolution is continuing to call for medical librarians to rethink their role in providing medical information public service librarians who challenge the traditional role of librarianship librarians have arrived at their current position through flexibility and the ability to adapt to changes librarians play a key role in the advancement of EBP librarians are in an ideal situation to become stronger partners in the improvement of health care growing academic librarian residency program for new librarians found the library with a pool of librarians lacking long-term experience in health sciences librarianship increase librarians� comfort in their new roles the first summer series, �Evidence Based Medicine for Health Sciences Librarians� the intangible rewards of working with other librarians, medical residents, and COM faculty were felt to be real and significant other questions remain to be answered by librarians and other educators involved in EBM

47

this increased visibility and stature places librarians in a position to affect evidence-based patient care decisions

51 EBMR helps to alert health care professionals, researchers, and librarians to clinically relevant and methodologically sound studies

52 it is critical that librarians become an integral part of the EBHC team review of the use of librarians� skills and of library and information services generally was undertaken by Barbara Cumbers the local librarian was not involved at any stage factors important to most firms were: support for searching and software problems, the length of time taken for document delivery, regular contact with librarian ten respondents felt that working more closely with a librarian would aid the use of research data in practice

53

nine librarians felt the same way this brings into question the role of the information professional, which will be considered later in this article the PRISE project is part of the Libraries and Librarian Development Programme managed by the Health Care Libraries Unit part of the Region�s Libraries and Librarian Development Programme the role of the information professional has already been intimated by a discussion of some of the current challenges in getting evidence-based practice into the primary and community setting

54

studies have shown the potential role of librarians in impacting upon patient outcomes in the clinical setting evidence-based healthcare agenda brings with it new challenges and opportunities for health librarians develop a Librarian of the Twenty-First Century programme

55

an orientation workshop on �Key roles of the librarian in evidence-based health care�

51

this paper provides an overview of EBP for librarians the parts of EBP that affect health librarians and the role that the health care literature plays in EBP are emphasized this has implications for both health care librarians and libraries librarians are in an ideal situation to become stronger partners in the improvement of health care

56

many opportunities and challenges exist for librarians and the development of EBP 57 awareness of the CRD and its services was very low outside of the librarian group

adapted from the NHS Regional Librarians Group Submission to the Functions and Manpower Review working with the NHS Regional Librarians Group in taking this forward to GPs librarians should also apply the principles of evidence-based medicine to their own profession

58

compiled by the Research Panel of the NHS Regional Librarians Group

52

Bibliography

Atlas, Michel C., et al. �Case Studies from Morning Report: Librarians� Role in Helping

Residents Find Evidence-Based Clinical Information.� Medical Reference

Services Quarterly 22.3 (2003): 1-14.

Bexon, Nicola and Louise Falzon. �Personal Reflections on the Role of Librarians in the

Teaching of Evidence-Based Healthcare.� Health Information and Libraries

Journal 20 (2003): 112-115.

Booth, Andrew, Alan J. O�Rourke, and Nigel J. Ford. �Structuring the Pre-Search

Reference Interview: A Useful Technique for Handling Clinical Questions�

Bulletin of the Medical Library Association 88.3 (2000): 239-246.

Dorsh, Josephine L., Susan Jacobson and Carol S. Scherrer. Teaching EBM teachers: a

team approach. Medical Reference Services Quarterly 22.2 (2003):107-14.

Palmer, Judith. �Health and Higher Education�a Migrant�s Tale.� SCONUL Newsletter

20 (2000): 19-23.

Plaice, Caroline and Pam Kitch. �Embedding Knowledge Management for NHS South-

West: Pragmatic First Steps for a Practical Concept� Health Information and

Libraries Journal 20.2 (2003): 75-85.

Timmermans, Stefan and Alison Angell. �Evidence-Based Medicine, Clinical

Uncertainty, and Learning to Doctor.� Journal of Health and Social Behavior

42.4 (2001): 342-59.

53

Williams, Linda and Lorri Zipperer. �Improving Access to Information: Librarians and

Nurses Team Up for Patient Safety.� Nursing Economics 21.4 (2003): 199-201.

Zandonella, Catherine. �Open Access: Will It Spell the End of the Medical Library?�

Medicine on the Net 9.11 (2003): 1-7.

Booth, Andrew, Alan J. O�Rourke, and Nigel J. Ford. �Structuring the Pre-Search

Reference Interview: A Useful Technique for Handling Clinical Questions�

Bulletin of the Medical Library Association 88.3 (2000): 239-246.