cadth webinar — finding the evidence
DESCRIPTION
These slides, presented at the CADTH webinar "Finding the Evidence: Tools and Techniques," show how CADTH Information Specialists use a variety of searching aids to perform comprehensive literature searches. This webinar was delivered on March 26, 2014.TRANSCRIPT
Webinar
March 26, 2014
About Us
• Amanda Hodgson ([email protected])
Manager, Information Services
• Hayley Fitzsimmons ([email protected])
Manager, Information Services
• Caitlyn Ford ([email protected])
Information Specialist
CADTH
CADTH is an independent, not-for-profit producer and broker
of health technology assessments. Federal, provincial, and
territorial health care decision-makers rely on our
evidence-based information to make informed policy and
practice decision about drugs and other health technologies.
www.cadth.ca
Agenda
• Role of the Information Specialist in HTA
• Introduction to “Finding the Evidence” website
• Creating your literature search
• Applying search filters
• Peer review of search strategies
• Grey literature searching
What are Systematic Reviews and Health Technology Assessments?
Scope of CADTH projects:
• Pharmaceuticals
• Medical, surgical and dental devices and procedures
• Diagnostics
Types of CADTH reports:
• Optimal Use reports
• Systematic reviews and economic evaluations
• Rapid reviews
• Horizon scanning reports
Role of the Information Specialist In
HTAs and Systematic Reviews • Experts in technical and methodological
issues of searching
• MLIS/MLS/MISt – educational requirements
• Work with the research team at every stage of review
WHY ARE THEY IMPORTANT?
• Help to mitigate potential biases in search
• Know search resources and how to use them
• Can advise on size and quality of search results
Standards for Systematic Searchers
• Finding what works in health care: standards for systematic reviews. Washington D.C.: Institute of Medicine of the National Academies; 2011.
• Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions. Version 5.1.0. [Updated March 2011]. The Cochrane Collaboration; 2011.
Major function of the Information Specialist Ensure researchers have all the available evidence on a topic
• failure to retrieve and analyze all relevant evidence on a topic can seriously compromise the quality of HTA and systematic review reports
However… • retrieving too much irrelevant literature can
seriously compromise the timeliness and costs of HTA and systematic review reports
Evidence in Medical Literature
Clinical
Practice
Guidelines
Health Technology
Assessments
(HTAs), Systematic
reviews, and/or
Meta-analysis
Source: Trip Database http://www.tripdatabase.com/
Safety
advisories
or
adverse
event
reports
Fundamental Challenge of Searching
Balancing Recall and Precision =
getting “all the evidence" while avoiding "junk"
http://www.creighton.edu/fileadmin/user/HSL/docs/ref/Searching_-_Recall_Precision.pdf
How Does This Translate Into Literature Searches?
Simple search for “diabetes” in Medline
Search Fields Number of Results
• Subject heading • Title, abstract, all free-text fields
400,000 +
• Title • 2+ times in abstract • Major subject heading
280,000 +
• Title • Major subject heading
160,000+
Solutions to the Recall versus Precision Conundrum?
• Finding the evidence: literature searching tools in support of systematic reviews http://cadth.ca/en/resources/finding-evidence-is
Creating YOUR Literature Search
• Understanding your research question
• PICO
• Scope for search terms
• Setting your parameters
What is your question?
P population
I intervention
C comparator
O outcome
C
Adapted with permission: Sampson, M.
Adapted with permission: Sampson, M.
Adapted with permission: Sampson, M.
Adapted with permission: Sampson, M.
Adapted with permission: Sampson, M.
Adapted with permission: Sampson, M.
Adapted with permission: Sampson, M.
Find controlled vocabulary and
keywords – always use both! Uncontrolled Terms (keywords in PubMed)
[ti] or [tiab] or [tw]
• Words in title & abstract fields
• Synonyms/Acronyms
• Spelling variants
• Old and new terminology
• Brand and generic names
• Lay and medical terminology
• Singular & plural variations
Controlled terms (in PubMed)
• MeSH subject headings [mh] or [major] or [mh:NoExp] or
[major:NoExp]
• Substance name field [nm]
• Pharmaceutical Action field
[pa]
• Registry number field [rn]
Population Intervention Comparison Outcome
Heart failure[mh] Exercise test
[mh] None
Meta-analysis[pt]
Randomized
controlled trial[pt]
(Cardiac OR heart
OR myocardial OR
congestive)
AND Failure
6MWT; SMWT;
Six minute; 6M;
6 Minute; 6Min;
6min
None Systematic
review; RCT;
CCT
Adapted from:http://www.oxfordradcliffe.nhs.uk/research/
researchers/news/documents/LiteratureSearchingGuidelinesChecklist.pdf
Keywords Keywords Keywords Keywords
Controlled terms Controlled terms Controlled terms Controlled terms
AND
O
R
Translating PICO
Search Translation
Could eliminate ‘test’
concept to be more
systematic!
“AND” with Systematic
Review plus RCT filters for
best evidence (represents
“clinical effectiveness” outcome)
Define Search Parameters
Major Medical Databases Medline/PubMed Embase PsycINFO CINAHL Cochrane Central
Focus Biomedical and health sciences
Biomedical and pharmaceutical database
Behavioural sciences and mental health
Nursing and allied health
Controlled clinical trials
Specialization Strong in clinical research; North American focus
Strong in pharmaceutical information; European focus; index conference abstracts
Includes fields such as psychiatry, management, business, education, social science, neuroscience, law, medicine, and social work
Includes fields such as nursing (clinical and theory), nurse education, physiotherapy, occupational therapy
Contains only citations of clinical trials; no need to use an RCT filter
Thesaurus MeSH EMTREE Thesaurus of Psychological Index Terms
CINAHL Headings
MeSH
Date Coverage Begins 1946 1974 1840 1981 circa 1966 (some earlier)
Journals Indexed 5,516 7,500 2,450 3,000 2,400 handsearched
Overlap with other databases
Yes Yes Yes Yes Yes
Translating the Search
CADTH PubMed cheat sheet
http://cadth.ca/media/i
s/Search-
dev/PubMed_Cheat_
Sheet_external.pdf
Visual Example of Typical Search Strategy
POPULATION
TERMS
INTERVENTION
TERMS
STUDY DESIGN
FILTERS
Types of Filters
• Specific methodology
– RCTs, Systematic reviews
• Clinical queries
– Diagnosis, Prognosis, Adverse events
• Population specific
– Humans, Pediatrics
• CADTH Filters http://www.cadth.ca/en/resources/finding-
evidence-is/string
Filters
http://www.cadth.ca/en/resources/finding-evidence-is/string
Filters: Warning
• No filter is infallible
• Filters are regularly being tested, tweaked and updated
but they almost always capture irrelevant records and
sometimes exclude relevant ones
RCT filter would pick up this article…
And this article…
“Randomized” in title, but not a study:
But not this one.
Study design NOT found anywhere
CADTH Peer Review Checklist for Search Strategies
http://www.cadth.ca/en/res
ources/finding-evidence-
is/peer-review-search-strat
CADTH Peer Review Checklist for Search Strategies
• Developed from: Sampson M, McGowan J, Lefebvre C,
Moher D, Grimshaw J. PRESS: Peer review of electronic
search strategies. Ottawa: Canadian Agency for Drugs
and Technologies in Health; 2008.
http://www.cadth.ca/media/pdf/477_PRESS-Peer-Review-
Electronic-Search-Strategies_tr_e.pdf
Peer Review: Does it Make a Difference?
• Recent research conducted to determine whether the
peer review of literature search strategies has an
effect on the number and quality of articles included in
CADTH rapid review reports
• What were the results?
Phase Two Results: Included Rapid Review Reports
EFFECT OF PEER REVIEW ON RETRIEVAL
19%
38%
43%
9/47 (19%) searches: No uniquearticles retrieved
18/47 (38%) searches: Unique articlesretrieved, none included in the finalreport
20/47 (43%) searches: Unique articlesretrieved, one or more articlesincluded in the final report
Getting Your Search Reviewed
• Colleagues
• Listservs
• Your librarian
• PRESS Forum website. A forum for librarians to request
and conduct peer reviews of search strategies.
http://pressforum.pbworks.com/
Best Practices for Search Strategies
1) Understand research question(s)
2) Use keywords and controlled vocabulary
3) Know database indexing and syntax
4) Use filters where appropriate
5) Have searches peer reviewed
6) Use caution: don’t overcomplicate
Why is Searching Grey Literature Important?
• Essential component of a comprehensive search
• Overcomes bias of commercial publications
• Part of evidence base
• Most HTAs created for healthcare decision-makers are
considered “grey literature” and are not found in traditional
databases
Grey Matters: A Practical Deep-Web Search Tool for Evidence- based Medicine
http://www.cadth.ca/en/resources/fi
nding-evidence-is/grey-matters
Health Technology Assessments (HTAs) KEY RESOURCES
• Databases:
– Centre for Reviews and Dissemination (CRD)
http://www.crd.york.ac.uk/crdweb/SearchPage.asp
– The Cochrane Library ($)
http://www.thecochranelibrary.com/view/0/index.html
– Trip Database
http://www.tripdatabase.com/
• Stand-alone websites:
– Health Quality Ontario Publications and OHTAC Recommendations
http://www.hqontario.ca/evidence/publications-and-ohtac-recommendations
– L’Institut national d’excellence en santé et en services sociaux (INESSS)
http://www.inesss.qc.ca/index.php?id=49&L=1
– Canadian Agency for Drugs and Technologies in Health (CADTH)
http://cadth.ca/en/products/health-technology-assessment
http://www.crd.york.ac.uk/crdweb/SearchPage.asp
AND
Safety/Advisory
HEALTH CANADA
• MedEffect Canada http://www.hc-sc.gc.ca/dhp-
mps/medeff/advisories-avis/index-eng.php
• CARN http://www.hc-sc.gc.ca/dhp-
mps/medeff/bulletin/carn-bcei_index-eng.php
• Canada Vigilance Online Database http://webprod3.hc-
sc.gc.ca/arquery-rechercheei/index-eng.jsp
Canada Vigilance Online Database
Canada Vigilance Online Database
Safety/Advisory
OTHER KEY RESOURCES
• U.S. Food and Drug Administration (FDA) MedWatch
http://www.fda.gov/Safety/MedWatch/default.htm
• European Medicines Agency (EMA) Patient Safety http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/landing/pha_listing.jsp&mid=WC0b01ac058001d126
Clinical Practice Guidelines
• A search for clinical practice guidelines may extend
to association websites or organizations dedicated
to a particular disease or patient group, depending
on the requirements of the search
– Guidelines
– Protocols
– Positions statements
– Patient information sheets and resources
– Recommendations
Clinical Practice Guidelines
KEY RESOURCES
• Canadian Medical Association CMA Infobase:
Clinical Practice Guidelines (CPGs)
http://www.cma.ca/clinicalresources/practiceguidelines
• National Institute for Health and
Clinical Excellence NICE Guidelines
http://guidance.nice.org.uk/index.jsp?action=find
• National Guideline Clearing House (NGC)
http://www.guideline.gov/
Follow us on:
@CADTH_ACMTS
Closing and Q&A