400863 lecture 12 understanding results 3 2014

8/10/2019 400863 Lecture 12 Understanding Results 3 2014

1/40

Foundations of Research and Evidence- Based Practice

Lecture 12

Understanding Results 3Nerida L. Klupp


2/40

Where are we up to.

Step 1. Is the result from data that is

Continuous

Categorical

Step 2. Is the result

Descriptive Inferential


3/40

Where are we up to

Step 3. If inferential, is the result about

Statistical Significance Clinical Significance

We will continue on with

clinical significance..


4/40

Minimum Important Difference

Minimum Important Difference (MID) is the

smallest worthwhile difference

(improvement) expected by a patient toproceed with a treatment


5/40

MID Example

Pain scale 0-10 (O is no pain, 10 is worst painimaginable)

You have heel pain and have marked this as 8

on pain scale Treatment options are

A) $10 comfy cushioned heel pad

B) $300 custom orthoses, daily strengthening andstretching

C) $5000 surgery, 6 weeks functional interruption,post-operative discomfort and some risk of harm


6/40

MID Example

What is the smallest improvement (points on apain scale) you would expect to make itworthwhile to have each treatment?

Treatment MID

A

B

C


7/40

What do you do with it?

You compare the treatment effect size with theminimal important difference. E.g. The amount ofimprovement in pain from having surgery for

spinal pathology

MID for having surgery= 80% improvement

95% Confidence Interval = Between 50% and 60%This means the best evidence suggests average improvement

in pain following this procedure is between 50% and 60%. The

patient does not proceed because their minimum

improvement wanted to undergo surgery is at least 80%.


8/40

Minimal important difference

At this stage you need to know!

What MID means

How it is used on a tree plot to determineclinical significance

To set an MID for a patient or populationevidence question requires further clinicalreasoning skills. You are not expected tosuggest MID values until third year.


9/40

So what is a tree plot?

The box represents the point estimate andthe line represents the confidence interval

Confidence Interval = 6-10kg

Mean = 8kg


10/40

treatment is not

worthwhile

Smallest clinicallyworthwhile effect or MID

treatment is

worthwhile

no effect

2 4 6 8 10 12 14 16-10 -8 -6 -4 -2 0

Tree Plot

Line of no effect

(no difference between groups)


11/40

treatment is not

worthwhile

Smallest clinically worthwhile

effect or MID: 5 kg weight loss

treatment is

worthwhile

no effect

Treatment effect = 8 kg weight loss following resistancetraining (95% CI 6 to 10) in favour of treatment group

2 4 6 8 10 12 14 16-10 -8 -6 -4 -2 0

Tree Plot


12/40

Clinical Significance

Statistical significance is that p-value < 0.05

Clinical significance is when the treatmenteffect (confidence interval) is equal or

more than the MID.


13/40

treatment is not

worthwhile



treatment is

worthwhile

no effect

2 4 6 8 10 12 14 16-10 -8 -6 -4 -2 0

Tree Plot

Is this result clinically significant?

Why?


14/40

If you put it all together

Difference in means =

The result that matters is the difference in weightloss between the groups (not within a group)

8kg (95%CI 6kg-10kg) p


15/40

But this is for continuousdata .

what do we do if the data is

categorical?


16/40

Revision

Continuous data involves an actualmeasurement e.g. average student mark onassignment = 62 marks

Categorical data involves a proportion of eventse.g. how many fails compared to passes = 5 failgrades compared to 10 pass grades

Both examples could provide information aboutstudent performance in an assignment


17/40

Categorical results

Categorical data are presented asproportions experiencing an event oroutcome of interest.

Risk ratio is a comparison of risks

Also called relative risk

Risk does not mean a harmful thing


18/40

TITANIC

SURVIVAL

ALIVE DEAD

FEMALE 308 154

MALE 142 709

Risk of death on Titanic

TOTAL

462

Females

851Males


19/40


The risk ratio can compare the probability ofdeath according to sex

The probability of a woman dying was 154

(women who died) out of 462 (total women)(154/462=0.33) or 33%.

The probability of a man dying was 709 (menwho died) out of 851 (total men)(709/851=0.8331) or 83%.


20/40


The risk ratio can compare the probability ofdeath according to sex

83% (male death proportion) compared to 33%

(female death proportion)= (0.8331/0.3333).

= There is a 2.5 greater probability of death

for males than for females.

Risk ratio = 2.5


21/40

Categorical results

Minimal important differences (MIDs) are alsoset in exactly the same way. How much morelikely do you want an event to occur to make anintervention worthwhile?

Confidence intervals are also the same e.g. RR =3 (95%CI 2.2- 3.8)

In the actual research the outcome was three timesmore likely, but in the population we can estimate with95% certainty that the outcome will be between 2.2and 3.8 times more likely


22/40

Example

The box represents the point estimate andthe line represents the confidence interval

Confidence Interval = 2.2 3.8

Risk Ratio = 3


23/40

No effect

One thing is different

Continuous: If there is no difference between

2 actual measures, the difference = 0 E.g. If both groups lose 5kg then the difference

between their improvements = 0

Categorical: If there is no difference betweenrisks, the difference value is 1


24/40


25/40

treatment is not

worthwhile

Smallest clinically worthwhileeffect or MID: 2.5 times greater risk

treatment is

worthwhile

no effect

Risk Ratio = 3 (95% CI 2.2 to 3.8) in favour of

intervention group

Minimal Important Difference = 2.5

1 1.5 2 2.5 3 3.5 4 4.5 50.001 0.01 0.1 0.5

Tree plot for categorical data


26/40

Clinical significance

A result is clinically significant if

The point estimate is larger than the MID

Most of the confidence interval larger than MID

A narrow confidence interval. If it is narrow, it is

called precise.


27/40

treatment is not

worthwhile



treatment is

worthwhile

no effect

Mean treatment effect= 2 kg in weight loss followingresistance training(95% CI -4 to 0) in favour of control group

group

2 4 6 8 10 12 14 16-10 -8 -6 -4 -2 0

Study A


28/40

treatment is not

worthwhile



treatment is

worthwhile

no effect

Mean treatment effect= 8 kg in weight loss followingresistance training(95% CI -2 to 18) in favour of treatment group

2 4 6 8 10 12 14 16-10 -8 -6 -4 -2 0

Study B


29/40

Activity

From the Study A and B on the previous two slides

1) Which study found the treatment effect wasclinically significant ? (the intervention works)

2) Which study found the treatment effect was not

clinically significant? (the control group did better-the intervention did not work)

3) Which study has the most precise results?


30/40

treatment is not

worthwhile

smallest clinically

worthwhile effect or

MIDvery harmful

treatment

very effective

treatment

treatment is

worthwhile

no effect

Study A

Study B

Study C

Tree plot of effect size


31/40

Systematic reviews

The results of systematic reviews arecalled forest plots

They plot multiple tree plots.

One tree plot = one clinical trial

Results will be either mean differences orrisk/odds ratios depending on type of data

A meta-analysis combines or pools thedata of the trials


32/40


33/40

Music for pain relief: Categorical

Categorical Forest Plot: Risk Ratio


34/40

Music for pain relief: Continuous

Continuous Forest Plot: Mean Difference


35/40

Tips on Reading Results

Dont skip the results section!!

Decide on what is your primary research question

Decide on your minimal important difference Find the point estimate and confidence intervals

for that question

Make sure they are for group comparisons

And ignore all the less relevant numbers!


36/40


37/40


38/40

Other studies

These lectures on results have onlydiscussed evidence on interventionswhatabout studies about diagnosis, prognosis,

and aetiology? They might use risk ratios

You will learn some other common results

for these types of studies next year.


39/40

Still to come

Final lecture

Lecture 14 Revision session on challenging topics

Exam advice

Bring any questions before your exam


40/40

Thank you!

400863 lecture 12 understanding results 3 2014

Documents