9.30am Choosing a Method of Qualitative Synthesis

Mr Andrew Booth,Reader in Evidence Based Information

Practice, ScHARR, University of Sheffield

Confusing Terminology, Variety of Choices

• Qualitative Systematic Review

• Qualitative Meta-Synthesis

• Qualitative Research Synthesis

Qualitative Evidence Synthesis

• Qualitative Interpretive Meta-Synthesis

• Best Fit Synthesis • Critical Interpretive Synthesis• Framework Synthesis• Meta-Aggregation• Meta-Ethnography• Meta-Interpretation• Meta-Narrative• Meta-Study• Meta-Summary• Narrative Synthesis• Qualitative Meta-Synthesis• Realist Synthesis• Thematic Synthesis

Pandora’s Box!

Overview of Approaches

Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Syst Rev. 2012 Jun 9;1:28. doi: 10.1186/2046-4053-1-28.

Bad Reasons for Choosing Method• Frequency of Use of

Method (e.g. Meta-Ethnography)

• Popularity/”Sexiness” of Method (e.g. Realist Synthesis)

• What a friend/ colleague/ mentor has used (once!)

• Bad experiences of others (may have been inappropriate!)

Good Reasons for Choosing Method

• Type of Question• Extent of Description versus Interpretation• Role of Theory• Type of Data• Intended Output• By Examining Methods Overviews and Published Examples

Other Considerations: Methodological Expertise in Team

Available Resources

Is Your Question……• Fixed? – Pre-defined as a

PICO (Population-Intervention-Comparison-Outcome) or SPICE (Setting-Perspective- Interest, Phenomenon of – Comparison- Evaluation) – Question is an “Anchor”

• (e.g. attached to an Effectiveness review)

• Negotiable? – To be explored as part of initial review process – Becomes clearer as you examine data (cp. Grounded theory approaches) – Question is a “Compass”

• NB. Even answering a fixed PICO question for HTAs may require exploration of phenomenon of untreated/pretreated condition (Lorenc et al, 2012)

Will You Describe or Interpret?

All Reviews figure on a continuum between Description and Interpretation• Description – What does

the data say? – factual reporting of “epidemiology” of studies, themes etc…

• Reader does work of interpretation

• Interpretation – What does the data mean? – “diagnosis” – subjective interpretation of “signs and symptoms” from data and themes etc…

• Reviewer does work of interpretation – may be contested

Aggregative, Interpretive or Integrative?

• Aggregative – (“Every study counts”) – for mapping, barriers and facilitators etc – e.g. Meta-Aggregation, Thematic Synthesis etc

• Interpretive – (Different perspectives, rounded out understanding) – to create models, possible explanations – e.g. Grounded Theory, Meta-Ethnography

Integrative – (Relating effectiveness and acceptability) – to answer what works, for whom, under what circumstances – e.g. EPPI-Centre Methods, Narrative Synthesis, Realist Synthesis

Will You Generate, Explore, Test Theory (Gough et al, 2012)?

• Generate – may require “suspension of disbelief” – quality assessment/ value judgement may come later (cp. Brainstorming)- Grounded Theory, Meta-ethnography

• Explore – looking for patterns - Narrative Synthesis, Thematic Synthesis

• Test – quality assessment differentiates well-supported and unsupported data - Framework Synthesis (incl. Best Fit Synthesis)

Decision to conduct a qualitative evidence synthesis

To interpret synthesised qualitative evidence and develop

explanatory theory or models

Purpose of the additional qualitative synthesis

or

Choice of Synthesis (Adapted from Noyes & Lewin, 2011)

To aggregate/ summarise/ integrate qualitative data to address questions in relation to a specific intervention review

Primarily to integrate and interpret qualitative and quantitative evidence within a single approach or integrated model. Can be used to develop explanatory theory.

Thematic analysis without theory generationMeta-aggregationMeta-summary

Product: Aggregated findings from source papers

Meta-ethnographyGrounded theoryThematic analysis with theory generation

Framework synthesis Realist Review

EPPI ApproachMatrix MethodNarrative SynthesisBayesian SynthesisCritical Interpretive synthesis

Product: Explanatory theory, analytical or conceptual framework or interpretative framework/mechanism

Best fit synthesis

How Rich (“Thick”) is Your Data?• Qualitative data from

“thin” studies (or textual responses to surveys) will not sustain interpretive approaches

• Limited to Meta-Aggregation, Thematic Synthesis, Framework Synthesis, Narrative Synthesis –type approaches

• Rich/“Thick” reports will sustain Meta-Ethnography/Grounded Theory – may allow selective sampling/ theoretical saturation

• NB. Is “Unit of Analysis” Individual Study (Meta-Aggregation, Thematic Synthesis) or “Body of Evidence” (e.g. Meta-Narrative or Critical Interpretive Synthesis approaches) or even Theory (Framework Synthesis/Best Fit Synthesis)?

What is Your Intended Output?• “output of some methods…

(Thematic Synthesis, textual Narrative Synthesis, Framework Synthesis, and ecological triangulation) is more directly relevant to policymakers and designers of interventions than…methods with a more constructivist orientation (Meta-Study, Meta-Narrative, Meta-Ethnography, Grounded Theory, CIS)…generally more complex and conceptual” (Barnett-Page & Thomas, 2009)

• Thematic Synthesis (including Meta-Aggregation) and Framework Synthesis produce findings that directly inform practitioners (Thomas & Harden, 2009)

• Interpretive approaches (e.g. CIS, Meta-Ethnography) produce a model that requires practitioners to interpret relevance and applicability to their own context

• Narrative Synthesis or EPPI-Centre (matrix) methods may help to integrate and present quantitative/qualitative work

What Expertise Can You Access?

• Expertise in Qualitative Research Methods (e.g. Grounded Theory; Framework Analysis, Thematic Analysis)

• Expertise in Synthesis Methods (incl. Searching, Data Extraction, Quality Assessment, Interpretation)

• Knowledge of Topic Area

Safest Options!

If…• There is a Pre-

existing Theory or Framework….

Then• ….Framework

Synthesis (including Best Fit Synthesis)

If…• There is No

Theory or Framework…

Then• …Thematic

Synthesis (Can also act as first stage of Meta-Ethnography)

If…• There is a

Proximate (Close-ish!) Theory or Framework….

Then• ….Best Fit

Synthesis

Some Examples

The Scenario

HTA Report – Systematic Review, Modelling & Qualitative Synthesis on Group Therapy for Postnatal Depression (UK)• Stevenson M, Scope A, Sutcliffe P, Booth A, et al. Group cognitive

behavioural therapy for postnatal depression: a systematic review of clinical effectiveness, cost effectiveness and value of information analyses. Health Technol Assess 2010;14(44).

• Scope, A., Booth, A. and Sutcliffe, P. (2012), Women’s perceptions and experiences of group cognitive behaviour therapy and other group interventions for postnatal depression: a qualitative synthesis. Journal of Advanced Nursing. doi: 10.1111/j.1365-2648.2012.05954

The Choice• Type of Question – Acceptability of Intervention• Extent of Description versus Interpretation – Factors Making Group

Therapy More or Less Acceptable (Descriptive) • Role of Theory – No Theory – Trying to separate Group Effect from

Therapy Effect• Type of Data – Very Thin Data, Small Number of Studies (n = 6) –

Descriptive Case Study Accounts in Nursing Journals• Intended Output – Alongside Effectiveness Review & Cost

Effectiveness Study – for ImplementationOther Considerations:

Methodological Expertise in Team – Novice Reviewer Available Resources – Limited Time in Comparison to “Main” Review

Choice = Thematic Synthesis

The Contribution

• While Group Therapy is Acceptable on Average there are some for whom it is unfavourable because:– Group Comparison – Some People are Getting

Better Quicker than I am– Group Comparison – Ms. X has been Coming Here

longer than I am and still is not any better• Identified Key Issue: Identification of those

Most Likely to Benefit.

The Scenario

Aim: To analyse meaning and motivation of the Wish To Hasten Death in patients with chronic illness or advanced disease• Monforte-Royo C, Villavicencio-Chávez C, Tomás-

Sábado J, Mahtani-Chugani V, Balaguer A (2012) What Lies behind the Wish to Hasten Death? A Systematic Review and Meta-Ethnography from the Perspective of Patients. PLoS ONE 7(5): e37117. doi:10.1371/journal.pone.0037117

The Choice• Type of Question – Explanatory• Extent of Description versus Interpretation – Factors that Help

to Explain a Phenomenon (Interpretative) • Role of Theory – Theory Generation• Type of Data – Contextually Rich – Qualitative Research Reports

– Thick Data [NB. 7 studies]• Intended Output – Stand Alone Research Report for

EnlightenmentOther Considerations:

Methodological Expertise in Team – Large Experienced Team with Qualitative Researcher

Available Resources – Extant Funded Project

Choice = Meta-Ethnography

The Contribution

• Wish to hasten death (WTHD) is multifactorial construct with multiple meanings that do not necessarily imply genuine desire to hasten one's death or actually taking steps towards this.

• Wish to hasten death (WTHD) is a phenomenon that appears, among patients in advanced stages of illness, as response to extreme suffering that affects all aspects of their human existence.

The Scenario

Aim: To evaluate benefits and constraints of collaborative health-related research by researchers and those affected by the issues under study and/or those who would apply research results and to examine how variations in programme context and mechanisms influence the process and outcomes of collaborative health intervention research? Jagosh, J., Macaulay, A. C., Pluye, P., Salsberg, J., Bush, P. L., Henderson, J., ... & Greenhalgh, T. (2012). Uncovering the benefits of participatory research: implications of a realist review for health research and practice.Milbank Quarterly, 90(2), 311-346.

The Choice• Type of Question – Explanatory• Extent of Description versus Interpretation – Factors that Help

to Explain Variation in Outcomes • Role of Theory – Theory Exploration• Type of Data – Contextually Rich – Quantitative and Qualitative

Research Reports – Thick Data [NB. Twenty-three PR partnerships described in 276 publications.]

• Intended Output – Explanatory Report for Multiple FundersOther Considerations:

Methodological Expertise in Team – Large Experienced International Team of Experienced Researchers

Available Resources – Extant Funded Project

Choice = Realist Synthesis

The Contribution• Used middle-range theory of partnership synergy to

demonstrate how PR can (1) ensure culturally and logistically appropriate research, (2) enhance recruitment capacity, (3) generate professional capacity and competence in stakeholder groups, (4) result in productive conflicts followed by useful negotiation, (5) increase quality of outputs and outcomes over time, (6) increase sustainability of project goals, and (7) create system changes and new unanticipated projects and activities.

• Generated new insights into benefits of PR regarding conflicts and negotiation between stakeholders, program sustainability and advancement, unanticipated project activity, and generation of systemic change.

References - 1• Barnett-Page E, Thomas J.

Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009 Aug 11;9:59.

• Booth, A, Papaioannou, D and Sutton, A J (2011). Systematic Approaches to a Successful Literature Review. SAGE publications

• Candy B, King M, Jones L, Oliver S. Using qualitative synthesis to explore heterogeneity of complex interventions. BMC Med Res Methodol. 2011 Aug 26;11:124.

• Dixon-Woods M, Agarwal S, Young B, Jones D, Sutton A. (2004) Integrative approaches to qualitative and quantitative evidence. London: Health Development Agency

• Gough, D, Oliver, S, Thomas J (2012) An Introduction to Systematic Reviews. London: Sage Publications.

• Lorenc, T., Pearson, M., Jamal, F., Cooper, C. and Garside, R. (2012), The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Res. Synth. Method, 3: 1–10.

References - 2• Popay J, Roberts H, Sowden A,

Pettticrew M, Arai L, Rodgers M, Britten N: Guidance on the conduct of narrative synthesis in systematic reviews. http://www.lancs.ac.uk/fass/projects/nssr/2007 .

• Pope C, Mays N, Popay J: Synthesizing Qualitative and Quantitative Health Evidence: a Guide to Methods. Maidenhead: Open University Press; 2007.

• Ring N., Ritchie K, Mandava L, Jepson R. (2011) A guide to synthesising qualitative research for researchers undertaking health technology assessment and systematic reviews. NHS Quality Improvement Scotland and University of Stirling, Edinburgh.

• Thomas J, Harden A (2009) Methods for the thematic synthesis of qualitative research in systematic reviews, BMC Medical Research Methodology 8:45