Improving efficiency and confidence in systematic

literature searching

Wichor M. Bramer - Gerdien B. de Jonge Erasmus MC - Rotterdam the Netherlands

EAHIL+ICAHIS+ICLC – Edinburgh (UK) – June 11nd / 12th 2015

MEDLINE (OVID)1 exp Psychotherapy/ (150715)2 Problem Solving/ (21235)3 psychotherap*.mp. (68147)4 ((cogniti* or family or behavior* or behaviour* or psychological*) adj5 (intervention* or treatment* or therap*)).mp. (107111)5 (problem* adj5 solv*).mp. (54029)6 CBT.mp. (5158)7 or/1-6 (265435)8 exp Parents/ (72641)9 exp Family/ (236222)10 Caregivers/ (21502)11 (parent* or mother* or father* or family or families or caregiver* or care-giver*).mp. (1252254)12 or/8-11 (1284976)13 exp Child/ (1543748)14 exp Infant/ (934743)15 Adolescent/ (1611500)16 (child* or infant* or adolesc* or baby or babies or toddler* or teenager* or youth*).mp. (3174766)17 or/13-16 (3174766)18 exp Pain/ (306198)19 exp Complex Regional Pain Syndromes/ (4336)20 exp Rheumatic Diseases/ (175661)21 exp Neoplasms/ (2561658)22 exp Diabetes Mellitus/ (314181)23 exp Asthma/ (105870)24 exp Brain Injuries/ (49346)25 exp Inflammatory Bowel Diseases/ (60373)26 exp Anemia, Sickle Cell/ (17799)27 exp Skin Diseases/ (810983)28 exp Genital Diseases, Female/ (358725)29 exp menstruation disturbances/ (24223)30 (pain* or headache*).mp. (582940)31 (rheumat* or arthriti* or fibromyalgia).mp. (232887)32 (cancer* or neoplas* or tumor* or tumour* or malignan* or carcinoma*).mp. (2952920)33 diabet*.mp. (471099)34 asthma*.mp. (140539)35 (brain adj5 (trauma* or injur*)).mp. (68497)36 (bowel* adj5 inflammatory adj5 (condition* or disease* or illness*)).mp. (31393)37 (sickle cell adj5 (disease* or disorder* or anemia*)).mp. (19955)38 ((skin adj5 (disease* or disorder*)) or eczema*).mp. (105706)39 ((gynecologic* or gynaecologic*) adj5 (disease* or disorder*)).mp. (5081)40 dysmenorrh*.mp. (5152)41 endometriosis.mp. (20182)42 Chronic Disease/ (219791)43 ((chronic* or longterm or long-term) adj5 (condition* or ill* or disease*)).mp. (418976)44 or/18-43 (5469388)45 randomized controlled trial.pt. (376608)46 controlled clinical trial.pt. (88576)47 randomized.ab. (297316)48 placebo.ab. (155203)49 drug therapy.fs. (1708731)50 randomly.ab. (214956)51 trial.ab. (308738)52 groups.ab. (1366808)53 45 or 46 or 47 or 48 or 49 or 50 or 51 or 52 (3363144)54 exp animals/ not humans.sh. (3954113)55 53 not 54 (2885271)56 7 and 12 and 17 and 44 and 55 (1119)57 (201203* or 201204* or 201205* or 201206* or 201207* or 201208* or 201209* or 201210* or 201011* or 201212* or 2013* or 2014*).ed. (2350810)58 56 and 57 (193)

EMBASE (OVID)1 exp Psychotherapy/ (188395)2 Problem Solving/ (26195)3 psychotherap*.mp. (96298)4 ((cogniti* or family or behavior* or behaviour* or psychological*) adj5 (intervention* or treatment* or therap*)).mp. (213343)5 (problem* adj5 solv*).mp. (62494)6 CBT.mp. (7776)7 or/1-6 (385750)8 exp Parents/ (155722)9 exp Family/ (335975)10 Caregivers/ (39951)11 (parent* or mother* or father* or family or families or caregiver* or care-giver*).mp. (1393196)12 or/8-11 (1438650)13 exp Child/ (2083396)14 exp Infant/ (860556)15 Adolescent/ (1219788)16 (child* or infant* or adolesc* or baby or babies or toddler* or teenager* or youth*).mp. (2974509)17 or/13-16 (3192664)18 exp Pain/ (829233)19 exp Complex Regional Pain Syndromes/ (7323)20 exp Rheumatic Diseases/ (188325)21 exp Neoplasms/ (3394555)22 exp Diabetes Mellitus/ (598732)23 exp Asthma/ (188864)24 exp Brain Injuries/ (122563)25 exp Inflammatory Bowel Diseases/ (3120)26 exp Anemia, Sickle Cell/ (26998)27 exp Skin Diseases/ (1197204)28 exp Genital Diseases, Female/ (490160)29 exp menstruation disturbances/ (51427)30 (pain* or headache*).mp. (971422)31 (rheumat* or arthriti* or fibromyalgia).mp. (348433)32 (cancer* or neoplas* or tumor* or tumour* or malignan* or carcinoma*).mp. (3603831)33 diabet*.mp. (709575)34 asthma*.mp. (213871)35 (brain adj5 (trauma* or injur*)).mp. (124711)36 (bowel* adj5 inflammatory adj5 (condition* or disease* or illness*)).mp. (43873)37 (sickle cell adj5 (disease* or disorder* or anemia*)).mp. (26819)38 ((skin adj5 (disease* or disorder*)) or eczema*).mp. (145349)39 ((gynecologic* or gynaecologic*) adj5 (disease* or disorder*)).mp. (18632)40 dysmenorrh*.mp. (9538)41 endometriosis.mp. (27781)42 Chronic Disease/ (149689)43 ((chronic* or longterm or long-term) adj5 (condition* or ill* or disease*)).mp. (512959)44 or/18-43 (7140484)45 random$.tw. (892218)46 factorial$.tw. (23405)47 crossover$.tw. (49675)48 cross over$.tw. (22262)49 cross-over$.tw. (22262)50 placebo$.tw. (203743)51 (doubl$ adj blind$).tw. (147609)52 (singl$ adj blind$).tw. (14595)53 assign$.tw. (241607)54 allocat$.tw. (84725)55 volunteer$.tw. (181433)56 Crossover Procedure/ (39305)57 double-blind procedure.tw. (219)58 Randomized Controlled Trial/ (347009)59 Single Blind Procedure/ (18444)60 or/45-59 (1428276)61 (animal/ or nonhuman/) not human/ (4623513)62 60 not 61 (1262127)63 7 and 12 and 17 and 44 and 62 (952)64 (201203* or 201204* or 201205* or 201206* or 201207* or 201208* or 201209* or 201210* or 201011* or 201212* or 2013* or 2014*).dd. (3247786)65 63 and 64 (263)66 limit 65 to embase (204)

0 5 10 15 20 25 30 35 40 45 500%

25%

50%

75%

100%

Meta-analysis (N=148)

Hours needed to create search strategy

Perc

enta

ge o

f Sys

tem

atic

Revi

ews

Hausner, 2015 (13); Saleh, 2014 (17); Lyon, 2014 (9); Gann, 2013 (47); Allen, 1999 (19) [1-5] and unpublished data from London Health Sciences Centre in Canada (43)

0 5 10 15 20 25 30 35 40 45 500%

25%

50%

75%

100%

Meta-analysis (N=148)

Bramer (N=258)

Hours needed to create search strategy

Perc

enta

ge o

f Sys

tem

atic

Revi

ews

Hausner, 2015 (13); Saleh, 2014 (17); Lyon, 2014 (9); Gann, 2013 (47); Allen, 1999 (19) [1-5] and unpublished data from London Health Sciences Centre in Canada (43)

How well do they perform?

Benchmark This study# terms 46 54# databases 3 7# hits 1399 2224# includes 19 25precision 1.2% 2.0%Time needed 960 min 70 min

Benckmark: 93 systematic reviews from 2014 with an acknowledgement or co-authorship of a medical information specialist from another Dutch academic hospital.

Systematic searching

1. Examining the research question (practicum)2. Identifying key terms (practicum)

a. Thesaurus termsb. Free text terms

3. Database / interface choice (discussion)4. Creating a basic search strategy (practicum)5. Optimizing the basic strategy (practicum)6. Translating the strategy between databases

(demonstration) 7. Increasing specificity or sensitivity (demonstration)8. Filters (discussion)9. Finding errors (demonstration)

Which articles can answer the question?

Facilitators and barriers in preconception care

evaluation studies of preconception care

Predictive factors for oronasal fistulas in children treated for clef lip

follow up studies of children treated for clef lip Study types not as exclusive filters

but additionally added in elements

Elements : Bias

Very specific characteristicsDuration of breastfeeding and intelligence

JudgementsPoor family functioning, harsh parenting and bullying behavior

Why a bias?

Elements : Duplicates

Pre-eclampsia AND pregnant AND women

Lichtenstein therapy AND inguinal hernias

Which elements to use:Does eating Broccoli help in the prevention of cancer

generalspecific

important

unimportant

Broccoli Cancer

Prevention

Broccoli Cancer Prevention

Which elements to use:your own research question

generalspecific

important

unimportant

PRACTICUM Describe the articles you want to find

Determine elements Plot them in the graph

Discuss with your neighbour

Identifying terms

Two major thesauri:MeSH (Medline)Emtree (Embase)

Three interfaces:PubMed (MeSH)Ovid (MeSH and Emtree)Embase.com (Emtree)

Identifying thesaurus terms

One element Multiple thesaurus terms (OR)

Multiple elements One thesaurus term

One element Multiple thesaurus terms (AND)

No thesaurus terms found

Truncate the search terms with * when searching MeSH database in PubMed

tear vs. tear*• Simple search in PubReminer• Use [tt] in PubMed to find translations• Use Google, Wikipedia etc. to find alternative terms

If all this fails:Use only free text words and see what you get

Identifying free text words

Optimal sensitivity only by additionally searching in title / abstract:• Relevant, non-inverted terms from synonyms /

entry terms• Relevant words / phrases from narrower terms

Add for yourself

- Abbreviations- Also partial abbreviations: Hip OA / Sentinel LN

- Alternative spelling- Randomization / Randomisation - Pediatric / Paediatric - Tumor / Tumour

PRACTICUM (5 min) – Finding keywords

Using the MeSH interface from PubMedTry to find as much search terms as possible for

Kidney dialysis

Renal Dialysis [MeSH]Renal Dialy* Hemodialy* Extracorporeal Dialy*Hemodiafiltration* Peritoneal Dialy*Kidneys, Artificial [mesh]Artificial Kidney*Blood Dialy*Acetate Free Biofiltrat* CAPDRenal Replacement Therapy[mesh:noexp]Renal Replace* Kidney Replace*

Haemodialy*Haemodiafiltrat*

(Dialysis [mesh] AND (Kidney Diseases [mesh] OR Kidney [mesh]))

Kidney dialy*

Database / interface choice

Advice for start database

1. embase.com if available2. Otherwise embase with proximity

(preferably Ovid)3. No embase? Medline with proximity4. No proximity interface available? PubMed

– more complicated – will miss relevant articles – or get much more noise

Creating a basic search strategy

Embase.com Ovid EBSCOHost PubMedBoolean AND, OR, NOTThesaurus '…'/exp exp "…"/ DE "…+" "…"[mh]Title / abstract

():ab,ti ().ab,ti. TI() OR AB() [tiab]

Proximity (() NEAR/n ()) (() ADJn ()) (() Nn ()) (() AND ()) truncated phrase*

Standard syntaxExtended version (20 syntaxes in 7 databases) as supplement in handouts

DEMO

Creating a basic search strategy

General rules:1. Work in a word document, not the interface2. Create single line search strategies

Single line approach or set numbers?Well structured?#1 brassica/exp#2 brassica*:ab,ti#3 broccoli:ab,ti#4 #1 OR #2 OR #3#5 neoplasm/exp#6 neoplas*:ab,ti#7 cancer*:ab,ti#8 tumor*:ab,ti#9 #5 OR #6 OR #7 OR #8#10 #4 AND #9#11 [animals]/lim#12 [humans]/lim#13 #11 NOT #12#14 #10 NOT #13

Unstructured?

(brassica/exp OR (brassica* OR broccoli):ab,ti) AND (neoplasm/exp OR (neoplas* OR cancer* OR tumor*):ab,ti) NOT ([animals]/lim NOT [humans]/lim)

Creating a basic search strategy

General rules:1. Work in a word document, not the interface2. Create single line search strategies3. Type code before adding the words4. After opening parentheses, immediately close them5. Never remove parentheses, only add6. Prepare proximity statement for reuse7. Copy terms from thesaurus instead of typing them8. Use arrow keys and Ctrl-shortcuts9. Use ab,ti to create the strategy, final syntax can be broad

PRACTICUM

Optimizing search strategy

Search temporarily per element: (thesaurus) NOT (title/abstract)(title/abstract) NOT (thesaurus)

Remove all field codes (automatic term mapping)

Check retrieved relevant articles for missed terms

DEMO + PRACTICUM

Translating the strategy between databases

Embase.com (Embase and Medline)Thorough optimization of thesaurus terms and title / abstract terms in search strategy

Macro and edit thesaurus

Medline (Ovid)MeSH terms in PubMed / Optimization

Add extra terms Remove thesaurus

Cochrane library

Macro

Web-of-science

Macro

Scopus

Macro

PubMed publisher

[sb]

Macro and edit

thesaurus

CINAHL (EBSCO)

edit thesaurus

PsycINFO (OvidSP)

Macro and edit

Google Scholar

Macro and edit

Lilacs / Scielo / ProQuest / Clinical trials etc

Translation

Ctrl-H: find and replacespecial tip: ^p for linebreak^t for tab

Macros: bit.ly/emcmacros

Considerations

Explode a broad term or not?Start exploded, when many irrelevant results appear,

be more specificTruncate a term or not?

Truncate as much as possible, hardly noisePhrases or proximity?

Always proximity (when available in interface), easier and more sensitiveHow broad should proximity be?

Start with 3, increase with 3 andevaluate newly retrieved articles

• Remove too general terms or too broad thesaurus terms.

• Do not explode thesaurus terms• Add an extra element• (mesh) AND (subheadings)

(mesh/subheading)• Narrow proximity• Separate elements proximity• Remove truncation (longer wordstem)• (I OR C) (I) AND (C)• Add Filters for study type?

date? language?

Only if you can afford missing relevant refs:• For the most important element: use major

and title instead of keyword and title abstract

Increase Sensitivity (always)

• Drop less important elements• Add extra words (thesaurus NOT tiab)• Generalize specific elements (broader

thesaurus terms)• Explode thesaurus terms• (mesh/subheading) (mesh) AND

(subheading)• Broaden proximity • Proximity separate words (AND)• Add truncation (shorten wordstem)• (I) AND (C) (I OR C)

• Temporarily broaden or drop a very specific element to optimize the others

• Save relevant hits on clipboard to check sensitivity of more specific searches

Increase Specificity (if too many)

Filters

Add filters only if they are important elements in your research question

(top right corner of graph)

Where to find good filters to use?Translating between databases?

Check for errorsHow do you know an error occurred?

Sometimes: More or less hits than expected

Too often: NOT!

Check your query for mistakes

Use Ctrl-F to search for ' AND '

Scan or search for Automatic Term Mapping (such as .mp. OR [all fields])

Successful search?

1. Coverage of accessible databases2. Features of search engine of used interfaces3. Quality of indexing4. Presence and quality of abstract

5. Quality of search and experience of searcher

Success factors / Time savers

• Optimization techniques • Macros for syntax translation • Direct feedback by researcher • Database order and interface choice • Single-line search strategies• Create query in Word and paste in database• Experience (be pro-active) • Having two screens on your computer• Computer literacy

keyboard short cuts (supplement 3)

Quality is the norm, not speed!

Not a blueprint for speed but for confidence by standardization

Speed can only be a result of experience

By gaining speed more customers can be helped and overall review quality will improve

Contact details

E-mail: w.bramer@erasmusmc.nlg.dejonge@erasmusmc.nl info.mb@erasmusmc.nl

Website: www.erasmusmc.nl/medbib

Twitter: @wichorLinkedIn: nl.linkedin.com/in/wichorSlideshare: www.slideshare.net/wichor

Further reading:• Bramer WM, Giustini D, Kramer BM, Anderson PF.

The comparative recall of Google Scholar versus PubMed in identical searches for biomedical systematic reviews: a review of searches used in systematic reviews. Syst Rev. 2013 Dec 23;2:115.

• Bramer WM, Holland L, Mollema J, Hannon T, Bekhuis T. Removing duplicates in retrieval sets from electronic databases: comparing the efficiency and accuracy of the Bramer-method with other methods and software packages (Eahil conference, Rome)

• Embase webinar : Systematic searching with Wichor Bramer (25 feb 2015)• Bramer WM, Pols DHJ, Bindels PJE, Bohnen AM. A fast, reliable and

objective method for creating thematic search filters in bibliographic databases [in process]

• Work in progress on coverage, retrieval and relevance ranking of the major databases for systematic reviews (>450 review searches; >100 finished; >4500 included references)

• Two articles in progress about this methodology (how to, and how well)

What to do with subheadingsSeparate element(broccoli/de OR (broccoli* OR bassica*):ab,ti) AND (neoplasm/exp OR (cancer* OR neoplas*):ab,ti) AND (prevention:lnk OR prevention/exp OR (prevent*):ab,ti)

Combined element(broccoli/de OR (broccoli* OR bassica*):ab,ti) AND (neoplasms/exp/dm_pc OR ‘cancer prevention’/de OR ((cancer OR neoplasm) NEAR/3 (prevent*)):ab,ti)

Sensitive More hits : 345 (101 unique) Less chance of missing relevant articles Recommended for start For SRs highly recommended

Specific Less hits : 257 (13 unique) Higher chance of missing relevant

articles To reduce the number of hits Use proximity in free text

Why start in embase?Source: Unpublished research Bramer (5048 included references from 111 reviews)

Relevance ranking

Which databases to useDatabase Used in % of our SRs

Embase.com 100%

Medline (Ovid) 99%

(PubMed 1%)

Cochrane central 98%

Web-of-Science 95%

Google Scholar 92%

PubMed (publisher[sb]) 91%

Scopus 36%

Cinahl 25%

PsycINFO 19%

Others (proquest, lilacs, scielo, sportdiscus, eric, amed etc)

9%

Unique articles per database

embase 509Web-of-Science 244medline 195Google Scholar 139Other databases 92

Source: own observation on 5048 included references of 111 systematic reviews