researching complex interventions. welcome, and thank you for coming we hope you enjoy it

ResearchingComplex Interventions

ResearchingComplex Interventions

Welcome, and thank you for coming

We hope you enjoy it

HELP

Chaos and complexity theories!

Not that bad, but it is about black boxes

Outline of Presentation

1. What makes an intervention complex?

2. Why are complex interventions important?

3. The MRC guidelines

4. An example of a complex intervention trial

5. This workshop

1) What is a ‘CI’?

Complex interventions are traditionally described as: “Interventions that contain

several interacting components”

Some Dimensions of Complexity

Some Dimensions of Complexity

1. Number of interacting components within the experimental and control interventions

Some Dimensions of Complexity

1. Number of interacting components within the experimental and control interventions

2. Number/difficulty of skills and behaviours required by those delivering or receiving the intervention

Some Dimensions of Complexity

1. Number of interacting components within the experimental and control interventions

2. Number/difficulty of skills and behaviours required by those delivering or receiving the intervention

3. Number of groups or organisational levels targeted

Some Dimensions of Complexity

1. Number of interacting components within the experimental and control interventions

2. Number/difficulty of skills and behaviours required by those delivering or receiving the intervention

3. Number of groups or organisational levels targeted

4. Degree of flexibility or tailoring of interventions permitted (individualisation)(and others - Lewin et al)

But some are more complex than others:

“Interventions built up of a number of components which may act both

independently and interdependently”

But some are more complex than others:

“Interventions built up of a number of components which may act both

independently and interdependently”

UNEXPECTED OUTCOMES

What is NOT a CI?

A drug trial?

What is NOT a CI?

A drug trial?

Is the whole RCT/EBM business really all about the evil multi-

national pharmaceutical industry?

2) Why are CIs important?

2) Why are CIs important?

Partly because they encompass all types of intervention other than

drug trials

Why are CIs important?

1. Chronic disease and health behaviours

Why are CIs important?

1. Chronic disease and health behaviours

2. The context in which an intervention is used

Why are CIs important?

1. Chronic disease and health behaviours

2. The context in which an intervention is used

3. In the real world we rarely use single interventions in isolation

Why are CIs important?

1. Chronic disease and health behaviours

2. The context in which an intervention is used

3. In the real world we rarely use single interventions in isolation

4. Unexpected consequences of interventions

1) Chronic Disease and Health Behaviours

Interventions that attempt to change behaviour are complex

Chronic Disease and Health Behaviours

We are not going to make inroads into chronic disease management unless we can learn how to change behaviours of health care professionals, patients and the public

Example: Diabetes

2) The Context

The Context

• The results of ‘simple’ trials may not be generalisable because of the influence of context on outcomes

• Context include many factors that can CHANGE over time

‘Macro’and ‘micro’-context effects

• ‘Macro’-context effects: Socio-economic and cultural circumstances of a trial and related cultural assumptions; the health care system; characteristics of population; how the problem is caused and sustained

• ‘Micro’-context effects: the environment in which the intervention is carried out, the interactions between HCP and patient

A ‘macro’ context effect

A ‘macro’ context effect

Healing rates of gastric ulcers, in trial ‘control’ groups

A ‘macro’ context effect

Healing rates of gastric ulcers, in trial ‘control’ groups

Germany: 62.4%Denmark: 16.7%

(Moerman D 2000)

A ‘micro’ context effect:the ‘placebo’ response

RCT effects

Intervention ‘Control’

(placebo)

Change

No Treatment

RCT effects (Kirsch, depression)

Intervention

Change

Adjust for no treatment

RCT effects (Kirsch, depression)

Intervention ‘Control’

(placebo)

Change

Adjust for no treatment

RCT effects (Kirsch, depression)

Intervention ‘Control’

(placebo)

Change

Adjust for no treatment

‘Medicine’ only worries about this bit

Assumption of the classic RCT

• That specific effects and context effects (characteristic effects and incidental effects) are divisible and additive

Specific effect and context effect

? SE

CE

Assumption of the classic RCT

• That specific effects and context effects (characteristic effects and incidental effects) are divisible and additive

• In many ‘complex interventions’ this is not the case (e.g. acupuncture and physiotherapy)

Paterson and Dieppe BMJ 2005

Interaction of specific effects and context effects

• Individualisation of therapy by adjusting what you do according to the response of the individual and their health beliefs

Interaction of specific effects and context effects

• Individualisation of therapy by adjusting what you do according to the response of the individual and their health beliefs

• What is generally done in clinical practice, especially by AHP and CAM practitioners. Regarded as a breach of intervention fidelity within the world of the traditional RCT – but can be OK in complex interventions

Trial ‘Fidelity’

• A principle of the classical RCT is that everyone gets exactly the same intervention

• This does not mirror ‘real life’

• During a complex intervention trial there may be variations in intervention delivery (by different people for example) and/or changes over time

This is OK as long as carefully recorded

3) Packages of Care

Packages of Care

• In chronic disease we rarely use interventions singly, we combine different modalities

Packages of Care

• In chronic disease we rarely use interventions singly, we combine different modalities

• NICE guideline committee on management of OA: Lots of trials of single interventions (mostly drugs that don’t help much), very few trials of packages of care (CIs)

Packages of Care

• In chronic disease we rarely use interventions singly, we combine different modalities

• NICE guideline committee on management of OA: Lots of trials of single interventions (mostly drugs that don’t help much), very few trials of packages of care (CIs)

• ERGO: we do not know what to recommend!

4) Unexpected Consequences

Unexpected Consequences

• Many trials of CIs have produced negative results when people thought the intervention was bound to work

• Some trials of CIs have produced unexpected positive and negative effects which were not understood

Complex Interventions can be seen as a ‘black box’

Complex Interventions can be seen as a ‘black box’

??

Complex Interventions can be seen as a ‘black box’

??

Emergent properties

Unexpected consequences

We need to unravel the ‘causal chain of understanding’

??

Emergent properties

Unexpected consequences

We need to unravel the ‘causal chain of understanding’

• This means assessing process as well as outcome

• This requires theory (what you think might happen and why) and may need both quantitative and qualitative approaches

3) MRC Frameworks for Complex Interventions

(Campbell et al BMJ 2000, Craig et al BMJ 2008)

The first framework (2000)

Preclinical Phase 1 Phase 2 Phase 3 Phase 4

Theory Modelling Exploratory RCT Implementation

Problems

• Model from drug development

• Linearity implied by the diagram

• Theory and modelling phases not well explained

• Implication that traditional RCTs are the preferred (only?) option for assessment

• Lack of attention to contextand others

The revised MRC framework for complex

interventions(Craig et al 2008)

Developing The Revised Framework

1. Workshop funded by MRC PHSRN, led by Paul Dieppe, Janet Darbyshire and Sally McIntyre, held in May 2006, c 30 invited attendees, with some of those who developed the 1st framework

2. Writing committee of PD, JD, SM + Peter Craig, Susan Michie, Irwin Nazareth, Mark Petticrew

3. Review by MRC HSPHRB

The revised framework1. Summary of what makes an intervention

complex, and the process of development and testing a CI

2. Questions researchers need to ask themselves

3. Examples of complex intervention studies using a wide variety of designs including non-randomisation/quasi-experimental studies

The ‘new’ model

The ‘new’ model

The ‘new’ model

Development:

1. Evidence base2. Theory3. Modelling process/outcome

Development:

1. Evidence base2. Theory3. Modelling process/outcome

Are you clear about what you are trying to do and why?

Feasibility/Piloting:

1. Testing procedures2. Recruitment/retention3. Determining sample size

Feasibility/Piloting:

1. Testing procedures2. Recruitment/retention3. Determining sample size

Can this work?Is it practical and sensible?What outcomes (interactions)?

Evaluation:

1. Assessing effectiveness2. Understanding process3. Cost-effectiveness

Evaluation:

1. Assessing effectiveness2. Understanding process3. Cost-effectiveness

What design are you going to use and why?

Implementation:

1. Dissemination2. Surveillance/monitoring3. Long-term follow-up

Implementation:

1. Dissemination2. Surveillance/monitoring3. Long-term follow-up

Publication is not enough!

4) An example of a CI Trial

1. The question – does physiotherapy work for people with isolated patello-femoral OA?

1. The question – does physiotherapy work for people with isolated patello-femoral OA?

2. The intervention – home based exercises and patella strapping with ‘education’ to aid behaviour change

1. The question – does physiotherapy work for people with isolated patello-femoral OA?

2. The intervention – home based exercises and patella strapping with ‘education’ and leaflets to aid behaviour change

3. Recruitment – from a community cohort

1. The question – does physiotherapy work for people with isolated patello-femoral OA?

2. The intervention – home based exercises and patella strapping with ‘education’ and leaflets to aid behaviour change

3. Recruitment – from a community cohort

4. The design – Modified Zelen design

Pre-screened: 178

Suitable for inclusion: 157

Consent obtained: 87

Pre-screened: 178

Suitable for inclusion: 157

Consent obtained: 87

Intervention: 43 No intervention:44

Pre-screened: 178

Suitable for inclusion: 157

Consent obtained: 87

Intervention: 43(38) No intervention:44 (44)

Final assessment: 82

Pre-screened: 178

Suitable for inclusion: 157

Consent obtained: 87

Intervention: 43(38) No intervention:44 (44)

Final assessment: 82

First consent

Second consent

Pre-screened: 178

Suitable for inclusion: 157

Consent obtained: 87

Intervention: 43(38) No intervention:44 (44)

Final assessment: 82

First consent

Second consent No trial consent

1. The question – does physiotherapy work for people with isolated patello-femoral OA?

2. The intervention – home based exercises and patella strapping with an educational package to aid behaviour change

3. The design – modified Zelen design

4. Recruitment – from a community cohort

5. Outcomes – ‘WOMAC’ pain and function

1. The question – does physiotherapy work for people with isolated patello-femoral OA?

2. The intervention – home based exercises and patella strapping with an educational package to aid behaviour change

3. The design – modified Zelen design

4. Recruitment – from a community cohort

5. Outcomes – ‘WOMAC’ pain and function

6. Process measures – quads strength, nested qualitative interviews

The ‘Result’

• Small difference in favour of intervention group at 5 months, disappeared by 12 months

Quilty et al – HTA Report 1997

Intervention

Control

0 5 m 12 m

Pain

The ‘Result’

• Small difference in favour of intervention group at 5 months, disappeared by 12 months

BUT:

The ‘Result’

• Small difference in favour of intervention group at 3 months, disappeared by 6 months

BUT:

• Huge variations in response - some a lot better some a lot worse; interviews showed that most people did not do physio (for good reasons); and that many thought they were a lot better, even if the WOMAC etc did not show that!

So how did we do with this trial?

Development

RATHER badly:

We did look at the literature to help design the intervention – BUT

1. No patient involvement in design2. No theoretical framework for behaviour

change aspect of the intervention3. No piloting of feasibility/acceptability4. No economic thinking or modelling

Feasibility/Pilot Stage

Quite badly:

1. We did not know if this would work before we started the trial – BUT:

2. We did work out the sample size

3. In practice the recruitment and retention went well (we were lucky)

Evaluation Stage

We did that quite well!

1. An innovative trial design that ‘worked’

2. Process measures to help unpack the black box of the intervention

3. Nested qualitative research to aid understanding

4. Good documentation of the intervention and how much individualisation was allowed

But we did not do any economics!

Implementation

Well, we told people about it

Well, we told people about it

And we published it

Well, we told people about it

And we published it

We published it more than once actually, and it helped several people’s careers ……

Well, we told people about it

And we published it

We published it more than once actually, and it helped several people’s careers ……

PATHETIC!

5) This Workshop

5) This Workshop

Is about methodology and how to do applied health research as much as it is about

complex interventions

5) This Workshop

Is about methodology and how to do applied health research as much as it is about

complex interventions

The MRC framework provides us with a window on all the key methodologies used in applied health research (Richards 2010)

5) This Workshop

1. Workshops

2. Research Cafes

3. Plenary Sessions

The ‘new’ model

The ‘new’ model

The ‘new’ model

HAVE FUN

HAVE FUN

BE NOISY

STAGE 4

STAGE 4:

Dissemination and Implementation

STAGE 4:

Dissemination and Implementation

Surveillance, Monitoring and Long Term follow up

STAGE 4:

Dissemination and Implementation

Surveillance, Monitoring and Long Term follow up

Evidence-Based Practice

Evidence-Based Practice

Trials

Evidence-Based Practice

Trials

Systematic Reviews

Meta-analysis

Evidence-Based Practice

Trials

Systematic ReviewsMeta-analysis

Guidelines/Dissemination

Evidence-Based Practice

Trials

Systematic ReviewsMeta-analysis

Guidelines/Dissemination

‘All will be well’

Evidence-Based Practice

Trials

Systematic ReviewsMeta-analysis

Guidelines/Dissemination

‘All will be well’

The Problem:Guideline Dissemination

does not work

The Problem:Guideline Dissemination

does not work

WHY?

The Problem:Guideline Dissemination

does not work

WHY?WHAT CAN WE DO TO

IMPLEMENT BEST PRACTICE?

Some VERY brief theoretical considerations

• The goal is ‘adoption’ or ‘normalisation’ of what is agreed best practice

• Changing things for the better means changing what individuals do, or changing how organisations work

Different Approaches

• Psychology – its about behaviour change: apply theories such as TPB (there are 33 of them!)

• Sociology – its about organisational structures, culture, interactions and practices: explore these

• Economics – its about money, preferences and choices: use incentives and financial levers

EXERCISE 1

Make a list of barriers and facilitators to the implementation

of the scenario you have

(Single words or short phrases)

Some ways of Disseminating and Implementing Research Findings

Some ways of Disseminating and Implementing Research Findings

1. Dissemination?

Dissemination

1. Academic publications and presentations

2. Stakeholder meetings and documents

3. Targeted one-page summaries for different groups

4. Working with patient groups

5. Simple recommendations for action

6. Politicians and the Press

Some ways of Disseminating and Implementing Research Findings

1. Dissemination?

2. Implementation?

Three Approaches

1. The ‘local champion’ model

2. The ‘Quality Circle’ Model

3. Appropriateness Criteria

1. A ‘Local Champion’

1. A ‘Local Champion’

Someone in the organisation, who knows how things work, and who does what, and is

passionate about the issue and about quality of health care delivery

1. A ‘Local Champion’

Someone in the organisation, who knows how things work, and who does what, and is

passionate about the issue and about quality of health care delivery

THIS IS A SYSTEM THAT WORKS WELL!

1. A ‘Local Champion’

Someone in the organisation, who knows how things work, and who does what, and is

passionate about the issue and about quality of health care delivery

THIS IS A SYSTEM THAT WORKS WELL!

BUT IT DEPENDS ON BEING LUCKY ENOUGH TO HAVE SUCH A PERSON

2. The ‘Quality Circle’

An approach Championed by Don Berwick and his Boston Based

‘Institute for Health Improvement’ (IHI Website)

2. The ‘Quality Circle’

2. The ‘Quality Circle’

Sharing best practice:

2. The ‘Quality Circle’

Sharing best practice:

1. Get groups together from different providers, each of whom has to provide ‘x’

2. The ‘Quality Circle’

Sharing best practice:

1. Get groups together from different providers, each of whom has to provide ‘x’

2. Discuss problem areas, successes and failures

2. The ‘Quality Circle’

Sharing best practice:

1. Get groups together from different providers, each of whom has to provide ‘x’

2. Discuss problem areas, successes and failures

3. Within provider groups decide which area(s) you need to improve on, use others’ work, decide how you are going to know if you have improved things

2. The ‘Quality Circle’

Sharing best practice:

1. Get groups together from different providers, each of whom has to provide ‘x’

2. Discuss problem areas, successes and failures

3. Within provider groups decide which area(s) you need to improve on, use others’ work, decide how you are going to know if you have improved things

4. Put new plan into practice

2. The ‘Quality Circle’

Sharing best practice:

1. Get groups together from different providers, each of whom has to provide ‘x’

2. Discuss problem areas, successes and failures

3. Within provider groups decide which area(s) you need to improve on, use others’ work, decide how you are going to know if you have improved things

4. Put new plan into practice

5. Share outcomes in meetings with other providers

P P P

A D A D A D

S S S

Plan, Act, Study, Do

LS1 LS2 LS3

AP1 AP2 AP3

2. Appropriateness Criteria

Developed by the Rand Organisation in California to deal with wide variations in provision of common procedures such as

coronary artery bypass

3. Appropriateness Criteria

3. Appropriateness Criteria

1. Search the literature to see what factors might affect decision to do ‘x’

3. Appropriateness Criteria

1. Search the literature to see what factors might affect decision to do ‘x’

2. Develop scenarios around doing ‘x’ in which you alter key variables (e.g. age)

3. Appropriateness Criteria

1. Search the literature to see what factors might affect decision to do ‘x’

2. Develop scenarios around doing ‘x’ in which you alter key variables (e.g. age)

3. Get a panel to rate scenarios on a 0-9 scale

3. Appropriateness Criteria

1. Search the literature to see what factors might affect decision to do ‘x’

2. Develop scenarios around doing ‘x’ in which you alter key variables (e.g. age)

3. Get a panel to rate scenarios on a 0-9 scale

4. Meet, discuss discrepancies, re-score

3. Appropriateness Criteria

1. Search the literature to see what factors might affect decision to do ‘x’

2. Develop scenarios around doing ‘x’ in which you alter key variables (e.g. age)

3. Get a panel to rate scenarios on a 0-9 scale

4. Meet, discuss discrepancies, re-score

5. Audit activity in ‘x’ against criteria

EXERCISE 2

Work out how you would implement the intervention outlined in your scenario

(assume cost-effectiveness)