Zoe G. DaviesCentre for Evidence-Based ConservationUniversity of Birmingham, UK
Systematic Review Protocol Development
Overview
What is a protocol?
What is the value of a protocol?
Question formulation
Components of a protocol
What is a Protocol?
The starting point
Explicitly states the methodology to be followed during the systematic review process
Available for peer-review by subject experts
What is the value of a protocol?
Formalises the question under review
Helps to avoid bias
Ensures transparency
Question: formulation
Define the systematic review question
Key elements:– Subject
(i.e., unit of study to which the intervention is to be applied)
– Intervention (i.e., policy or management intervention under scrutiny)
Question: formulation
– Outcome(i.e., any measured outcome that can be used to judge the effectiveness of the intervention)
– Comparators
(e.g., intervention vs. no intervention or intervention y vs. intervention z)
Question: making it relevant
Dependent on the purpose of the systematic review– Management, policy or research driven question
Consultation with interested stakeholders, end-users and subject experts
Striking the balance– Not too broad, not too specific
Question: secondary objectives
Identify factors that may influence the outcome of studies
Also referred to as:– Reasons for heterogeneity – Effect modifiers (meta-analytical terminology)
Case Study: hedgerow corridors
Evaluate the effectiveness of habitat corridors in promoting population viability of target species and biodiversity within fragments of remnant habitat
20 years of debate within the ecological literature
Case Study: hedgerow corridors
Do hedgerows mitigate woodland habitat fragmentation?
Do hedgerows increase the population viability of target species occupying otherwise isolated fragments of woodland habitat?
Case Study: hedgerow corridors
Key question elements:– Subject: mammal, bird, invertebrate
or amphibian populations or assemblages
– Intervention: a hedgerow, or hedgerow network, connecting two or more woodland habitat
fragments
Case Study: hedgerow corridors
– Outcome: desired primary outcomes were change in population density for a target species or change in species richness within assemblages
– Comparator: No comparator was necessary for inclusion (although appropriate spatial or temporal controls were a prerequisite for studies to be included in any subsequent meta-analysis)
Case Study: hedgerow corridors
Reasons for heterogeneity: – Physical structure of the hedgerow
– Vegetation composition of the hedgerow
– Nature of the non-habitat matrix
– Life history stage of the target species (e.g., dispersing juveniles)
Protocol: background
Rational behind the systematic review
Put the review question into context – Woodland fragmentation – Habitat connectivity– Hedgerow ecology
Protocol: literature search strategy
State how and which information sources will be searched
Key words:– Reflect the population, intervention and outcome
– Consider synonyms, alternative spellings and abbreviations(e.g., colonise and colonize)
– Foreign language translations
Protocol: literature search strategy
Hedgerow* AND corridor* Hedgerow* AND movement* Hedgerow* AND dispersal Hedgerow* AND colonisation Hedgerow* AND colonization Hedgerow* AND connectivity Hedgerow* AND population* Hedgerow* AND communit*
Protocol: literature search strategy
Combinations and permutations– Trade off between effort and return– Sensitivity vs. specificity
Scope searches and refine
Search generic and specific information sources
Protocol: literature search strategy
Online databases and libraries– Scientific literature databases (e.g., WOK and JSTOR)– Statutory and non-governmental organisation websites
(e.g., Defra, EN and RSPB)– Internet search engines (e.g., Dogpile and Google Scholar)
Specialists in the field
Bibliographies
Hand searching libraries and museums
Protocol: study inclusion criteria
Based on the key elements of the question
State the filtering process to be used– Title– Abstract
(Assessed by a second review and test for agreement)
– Full text
(Assessed by a second review and test for agreement)
Protocol: study quality assessment
Hierarchy of evidence– RCT’s– Control trials without
randomisation– Site comparisons– Time series data
Used to determine study quality thresholds for included articles– Dependent on the purpose of the review
Protocol: data extraction
What data needs to be extracted from the accepted studies?
– Pilot data extraction forms
– Contact authors or organisations for retrieval of missing data
– Database or spreadsheet of all information relevant to the review
Protocol: data synthesis
How are the studies to be pooled?– How will differences in the studies be taken into
account?
Propose analyses to be undertaken
May not be possible to be specific at the protocol stage
Protocol: timescale
Set out key milestones
(e.g., completion of searching, study selection, etc.)
Possible delays
(e.g., consultation periods, inter library loans, etc.)
Some stages may overlap
Protocol: modifications
Amendments to the methodology(e.g., if there are no studies which meant the inclusion
criteria)
Modifications must be documented and justified– Maintain transparency– Allow independent parties to judge review validity
Further Information
Available from our website: – www.cebc.bham.ac.uk
Medical systematic review centres:– The Cochrane Collaboration
(www.cochrane-net.org)– NHS Centre for Reviews and Dissemination
(www.york.ac.uk/inst/crd)