Focusing the question

Janet HarrisCochrane Qualitative & Implementation

Research Methods Group

ESQUIRE Qualitative Systematic Review Workshop 2015

Session aims

• To provide examples of question formulation for qualitative and mixed methods reviews

• To identify the issues and challenges in formulating different types of review questions

What are qualitative research questions?• What are people’s experiences of the intervention?

• What outcomes or aspects of the intervention do they value and why?

• Why do interventions work (or not work)? For whom, in what circumstances? At what point in time?

• How does the surrounding context hinder or promote implementation of a programme, service or treatment?

• How can a particular intervention be adapted for large scale roll out? (Roen 2006)

What are the aims of using qualitative research in reviews?

• Reviews using qualitative research, or integrating qualitative and quantitative research are conducted in order to:1) Inform the development of a review protocol

2) Explain relative effectiveness or variations in effectiveness of an intervention

3) Produce an explanatory theory, conceptual model, or interpretive framework

1) Asking qualitative questions to develop a review protocol

• First: what is a review protocol?• Helps to plan the review and anticipate

problems

• Encourages comment and review

• Aids transparency and audit

• Can be prescriptive or allow for iteration

Elements of a review protocolElements• Title

• Background

• Review question/objectives

• Inclusion and exclusion criteria

• Search strategy

• Screening

• Quality assessment

• Coding and data extraction

• Synthesis methods

• Conflicts of interest

Contents

• Describes problem and need for review, directly linked to review questions

• Decisions on what to include and exclude are decided after scoping available research

• Decisions on how to assess quality are based on types of studies found during scoping, and types of synthesis that can be done based on existing studies

Framing background questions

• The aim of background questions is to collect general information about an issue, a phenomenon, an intervention

• Background questions are useful when• Little is known about a subject• Different members of the team have different levels of

knowledge• The problem is being framed in different ways• Those who are commissioning the review have information

needs that may not match what has been researched

Caught between a rock and a hard place

://www.youtube.com/watch?v=AB7VF980cEA

Background question examples

• Little is known:• Involving patients and the wider community in diabetes

research

• Different problem frames:• Do social support interventions to reduce hospital

admissions

• Unrealistic information needs:Factors influencing the effectiveness of community-based

malnutrition programmes versus hospital-based programmes

Background question format

• Problem• Root question: W5H

• Who? • What?• Where? • When? • Why?• How?

Background question example 1• Problem:

• Lack of involvement in designing diabetes interventions may produce inappropriate interventions

• Root question: W5H• Who has involved patients and communities in diabetes

research? • What sort of involvement has been used?• Where: in which countries? • When: At what stage(s) of designing and conducting the

research was involvement used? • Why was involvement used – what were the benefits?• How were people involved?

Background question example 2

• Problem: Frail elderly people • Lack social support to manage conditions• Represent over 50% of avoidable hospital admissions

• Root question: W5H• Who provides social support for self management? • What sort of support is needed/provided?• Where can support be accessed? • When is support provided? • Why?• How is social support related to a reduction in avoidable

admissions?

Background question: Your turn

• Problem: Pick a problem in your group (2 minutes)

• Root question: Lets develop together• Who? • What?• Where? • When? • Why?• How?

Defining the scope of your review

• Background questions are one way to define/negotiate the scope of your review

• The scope may also be defined by the type of review• For example, reviews aiming to explain the

effectiveness of an intervention could take issues in the original intervention review as a starting point

2) Question formulation to explain effectiveness of an intervention

• Read Glenton et al 2013

• Why is a qualitative review being conducted?

• What is the phenomenon of interest for the qualitative review?

• Can you formulate a qualitative review question?• Let’s look at several question formulation tools

Question formulation tool for qualitative evidence synthesis

S Setting – Where? In what context?

P Perspective – For who?

I Intervention (Phenomenon of Interest)– What?

C Comparison – What else?

E Evaluation – How well? What result?

(Booth 2006)

Question formulation for health managementE Expectation—what does the search requester want the information for (the original ‘I’s)?

C Client Group.

L Location.

I Impact—what is the change in the service, if any, which is being looked for? What would constitute success? How is this being measured?

P Professionals.

S Service—for which service are you looking for information? For example, outpatient services, nurse-led clinics, intermediate care.(Wildridge & Bell 2002)

Developing qualitative questions for intervention reviews• Qualitative research should be considered if

• Your review question is about a complex intervention

• More information is needed on the components of an intervention – what are various elements of the intervention how do they work together?

• Information is needed on the surrounding context – the setting, the environment, that may influence the relative success of the intervention

• Information is needed on why the intervention works or doesn’t work in different settings, for different populations

Systematic review questions and approaches to conceptualizing interventions

Systematic Reviews of Complex Multicomponent Health Care Interventions [Internet].Guise JM, Chang C, Viswanathan M, et al.Rockville (MD): Agency for Healthcare Research and Quality (US); 2014 Mar. http://www.ncbi.nlm.nih.gov/books/NBK194844/ [Accessed 19 August 2014]

3) Qualitative questions for reviews that produce models or theories

• When the aim is to • identify components of an intervention,

or intervention types, or processes of implementation

• map different definitions or dimensions of a concept

• produce an explanation for how and why something works

Examples of questions to develop models and theories

• What are the elements in a causal pathway between advice interventions and health outcomes? (Allmark et al, 2013)

• How does community based peer support impact on understanding of existing health information and use of health information and health services to improve health and reduce health inequalities? (Harris et al 2015)

Aim of the review: Producing a logic model or framework• The question aims to

• identify types of interventions

• Types of outcomes from the intervention

• Relationships between them

• The model can be used to build hypotheses, and/or decide the focus of a review based on amount of evidence

Aim of the review: Producing explanatory theory• The question aims to identify

concepts and constructs in primary studies that contribute to building an explanation for a phenomenon• What is peer support?

• What is community-based peer support?

• How is support related to health inequalities?

Question formulation exercise

• Can you formulate a review question that aims to develop a logic model or theory?

References

Allmark, P., Baxter, S., Goyder, E., Guillaume, L. and Crofton-Martin, G. (2013) Assessing the health benefits of advice services: using research evidence and logic model methods to explore complex pathways. Health & Social Care in the Community, 21 (1). 59 - 68. I

Booth, A. (2006). Clear and present questions: Formulating questions for evidence based practice. Library Hi Tech, 24(3), 355-68. doi:10.1108/07378830610692127

Guise JM, Chang C, Viswanathan M, et al. Systematic Reviews of Complex Multicomponent Health Care Interventions [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2014 Mar. http://www.ncbi.nlm.nih.gov/books/NBK194844/ [Accessed 19 August 2014]

Harris J, Springett J, Croot L, Booth A, Campbell F, Thompson J, et al.Can community-based peer support promote health literacy and reduce inequalities? A realist review. Public Health Res 2015;3(3)

Wildrige V & Bell L. How CLIP became ECLIPSE: a mnemonic to assist in searching for health policy/management information; HILJ 2002, 19 (2) 113-5 DOI: http://onlinelibrary.wiley.com/doi/10.1046/j.1471-1842.2002.00378.x/full