![Page 1: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/1.jpg)
Leeds Institute of Health Sciences
Getting better evidence
Stephen Morley
![Page 2: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/2.jpg)
Why bother with trials … ?
From: Moore & McQuay ‘Bandolier’s little book of making sense of the evidence’ 2006
Knowledge
Wisdom
Sys Reviews& Meta-anal
Evidence in clinicalpractice
DistillationIntegration
Quality
ExperienceValues
Conditions
InformationSingle RCTs
![Page 3: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/3.jpg)
Trials
Good Poor
ReviewsGood Ideal May
helpPoor Can
repeatWill
mislead
From: Moore & McQuay ‘Bandolier’s little book of making sense of the evidence’
![Page 4: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/4.jpg)
Issues
• Trial quality– Design, size matters– Quality and effect size
• Outcomes– Variety, validity and ‘clinical’ relevance– Efficacy and effectiveness
• Treatment content and coherence– Is there a model?– Mediation?
![Page 5: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/5.jpg)
Trial quality
![Page 6: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/6.jpg)
Cumulative trials over years
Hoffman et al 2007
Morley et al 1999
Words of caution …
What to count?Quality …Content …
CBT on the label ‘may not be CBT in the tin’
What is CBT in this context ?
![Page 7: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/7.jpg)
Tools for assessing quality
Ideal May help
Can repeat
Will mislea
dFilter out poor quality trials by setting cut-offs
Investigate influences of feature on
conclusions – Meta-regression
WHY?
Trials
+ -
+
MA
-
![Page 8: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/8.jpg)
Quality scales: The Jadad Scale
1. Is the trial randomised (1 point)+1 point if method described and appropriate
2. Is the trial double blind (1 point)+1 point if method given and appropriate
3. Is there a description of withdrawals and drop outs (1 point)
Suggested cut-off = 3
It’s simple Quick Captures major biases Can be reliable with basic training
But
Criterion 2 eliminates all complex interventions
Doesn’t capture important features of psychological trials
![Page 9: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/9.jpg)
Quality scale for psychological trials
Yates et al, Pain 2005: 117; 314-325
Identify and recruitDelphi panel
Panel generates and agreesItems: 3 rounds
Expert panel writes QS
Reliability and validity studies using novice and expert raters
Datafrom 31
published trials
Final QS
![Page 10: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/10.jpg)
Quality scale for psychological treatment trials
32 parts
Is there a good description of the sample in the trial?
Sample characteristics 0 1
Group equivalence 0 1
44 parts
Have adequate steps been taken to minimise biases?
Randomisation 0 1 2
Allocation Bias 0 1
Measurement Bias 0 1
Treatment expectations 0 1
Reliability
ICC absolute agreement
Full scale > 0.9
Treatment quality > 0.9
Design quality = 0.85
Kappa for items
range 0.0 to 0.74
Agreement coefficient for items
>80%
![Page 11: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/11.jpg)
Strengths and weakness of psychological trials
Yates et al, Pain 2005: 117; 314-325
Design
Treatment
![Page 12: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/12.jpg)
Quality over time – the good news
Morley, Eccleston & Williams, unpublished
![Page 13: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/13.jpg)
Effect size and quality
Yates et al, Pain 2005: 117; 314-325
TotalQSβ = -.35 , p =.057
Treatment QS ns
Design QS, β = -.4 , p <.05
![Page 14: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/14.jpg)
Size matters
Data from Hoffman et
al 2007
![Page 15: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/15.jpg)
Outcomes
![Page 16: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/16.jpg)
Outcomes
underlying scale
x►
y►
z►►
x►
dysfunctional or clinical sample
functional or normal sample
a bc
►
CSC criteria
Morley in McQuay et al ‘Systematic reviews in pain research’ 2008, IASP press
![Page 17: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/17.jpg)
Turning continuous outcomes into dichotomous ones
0
5
10
15
20
25
30
35
40
0 5 10 15 20 25 30 35 40
Pre-treatment
Po
st-
treatm
en
t
Deteriorated from pre-treatment good functioning
Reliably deteriorated
Reliable improvement but not clinically significant
Reliable and clinically significant improvement
No reliable change
Reliable improvement but not clinically significant
Reliably deteriorated
![Page 18: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/18.jpg)
Heterogeneity of outcomes in trials
0
5
10
15
20
25
30Pain experience
Mood/ affect
Social role
Cognitive
Behavioural activity
Biological
Health care use
Miscellaneous
Number oftrials usingthe measure
Mean numberof measuresper trial
Data from Morley et al, Pain 1999: 80; 1-13
IMMPACT core outcomes
1. Pain
2. Physical Functioning (interference/disability)
3. Emotional functioning
4. Global improvement
5. Symptoms/adverse effects
6. CONSORT data
Dworkin et al, Pain 2005: 113; 9-19
![Page 19: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/19.jpg)
Stakeholders and outcomes:who wants what change?
0
5
10
15
20
25
30Pain experience
Mood/ affect
Social role
Cognitive
Behavioural activity
Biological
Health care use
Miscellaneous
Health care provider
Researcher
The patient
Employers
What outcome do you want?
Sleep
Weakness
Fatigue
Emotional well-being
Enjoyment of life
Doing tasks
IMMPACT, 2008 Pain:137; 276-285
![Page 20: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/20.jpg)
How much change do you want?
Mdn % change desired
ES(d) %meeting RCI
%meeting CSC
Severity 60 1.45 61.2 20.9
Impact 75 1.70 73.2 57.7
Interfere 66 1.82 75.0 63.2
Activity 44 1.38 36.8 16.2
Thorne & Morley in preparation
![Page 21: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/21.jpg)
How much change do you want?
Thorne & Morley in preparation
Interference Pain severity
![Page 22: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/22.jpg)
The evidence cycle
Efficacy studiesRandomisedControlled
Trials
Evidence-based practiceas policy
Practitioners
Practice-based evidence
Effectiveness studiesRoutineClinical
Treatment
Practitioners
![Page 23: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/23.jpg)
Practice based evidence
Morley, Williams & Hussain, Pain 2008; 137: 670-680
![Page 24: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/24.jpg)
Outcome categories - efficacy
8 58
3 5 4
20 6 20
Crude NNT values
Morley, Williams & Hussain, Pain 2008; 137: 670-680
![Page 25: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/25.jpg)
Benchmarking from RCT data
From Minami et al J Consult Clin Psychol 2007;75: 232-43
![Page 26: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/26.jpg)
Effectiveness + benchmark
WLC group
Tx
Group
RCT is: Williams et al. Pain 1996;66(1):13-22.
![Page 27: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/27.jpg)
Treatment
![Page 28: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/28.jpg)
Quality controlling treatment
• Manuals – protocols?• Training for therapist and teams?• Supervision?
• Patient monitoring systems?
![Page 29: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/29.jpg)
Is there a model: what’s in the tin?
Unpublished data from Morley et al, Pain 1999: 80 1-13
CBT treatment components across trials
![Page 30: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/30.jpg)
What’s the model?
• Generic– Principles of engagement and delivery
• Collaborative, information provision / education – Changing key ‘cognitive appraisals’ through behavioural
experimentation– Techniques: principled application or self service store?
• In PMP/CPM programmes embedded within pharmacotherapy, functional restoration, medical management– How coherent and integrated are they?
• Developing more specific models? A debate (JV)
![Page 31: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/31.jpg)
Thanks to …
• Chris Eccleston• Amanda Williams• Henry McQuay• Andrew Moore
• Wendy Callaghan
• Johan Vlaeyen• Lance McCracken
• Shona Yates• Sumerra Hussain• Fiona Thorne
![Page 32: Leeds Institute of Health Sciences Getting better evidence Stephen Morley](https://reader038.vdocuments.us/reader038/viewer/2022110116/551701d8550346f5558b5003/html5/thumbnails/32.jpg)
Seen in Leeds ….
PAINis just
weakness
leaving
the
BODY