quantitative presentation

7/28/2019 Quantitative Presentation

1/39

Critical appraisalof quantitative

researchNovember 2010

Sarah Lawson

Research & Learning Support

[email protected]

1
mailto:[email protected]:[email protected]


2/39

Learningobjectives Understandtheprinciplesofcriticalappraisalanditsrole

inevidencebasedpractice

Understandthedifferentlevelsofquantitativeevidence

Be

able

to

appraise

quantitative

research

and

judge

its

validity

Beabletoassesstherelevanceofquantitativeresearchtotheirownwork

Knowaboutresourcesavailabletohelpthemtocriticallyappraiseresearch

Beabletocriticallyappraisequantitativeresearchasagroup


3/39

Whatisevidencebasedpractice?

Evidencebasedpracticeistheintegrationof

individualclinical

expertise

withthe

best

available

external

clinical

evidence fromsystematicresearchand

patientsvalues

and

expectations


4/39

Decisionorquestionarisingfroma

patientscare.

Formulateafocusedquestion.

Searchfor

the

best

evidence.

Appraisetheevidence.

Applythe

evidence.

Theevidencebasedpractice(EBP)

process.


5/39

EBPinpractice

depending

upon

speciality,

between

50

and

80

per

centofall'medicalactivity'isevidencebased.

www.shef.ac.uk/scharr/ir.percent.html
http://www.shef.ac.uk/scharr/ir.percent.htmlhttp://www.shef.ac.uk/scharr/ir.percent.html


6/39

Whydoesevidencefromresearchfail

toget

into

practice?

75%cannotunderstandthestatistics

70%cannot

critically

appraise

aresearch

paper

Using

research

for

Practice:

a

UK

experience

of

the

barriers

scale.

Dunn,V.etal.


7/39

Whatiscriticalappraisal?

Weighingupevidencetoseehowusefulitisindecisionmaking

Balancedassessmentofbenefitsandstrengthsofresearchagainstitsflawsand

weaknesses Assessresearchprocessandresults

Skillthatneedstobepracticedbyallhealthprofessionalsaspartoftheirwork


8/39

WhatcriticalappraisalisNOT

Negativedismissal

of

any

piece

of

research

Assessmentofresultsalone

Basedentirely

on

statistical

analysis

Onlytobeundertakenbyresearchers/

statisticians


9/39

Whydoweneedtocritically

appraise? Itusuallycomesasasurprisetostudentsto

learnthat

some

(the

purists

would

say

99%

of)

publishedarticlesbelonginthebinandshould

notbeusedtoinformpractice

Greenhalgh 2001


10/39

Whydoweneedtocritically

appraise? studieswhichdon'treporttheirmethodsfully

overstatethebenefitsoftreatmentsbyaround25%

Khanetal.ArchInternmed,1996;Maheretal,Lancet1998.

studiesfunded

by

apharmaceutical

company

were

foundto be4timesaslikelytogiveresultsthatwerefavourabletothecompanythanindependentstudies

Lexchin etal,BMJ,2003


11/39

Sourcesofbias

poorcontrolgroup/controldosage

surrogateoutcomes

ignoredropouts

modifytriallength

misusebaselinestatistics

statisticsoverload


12/39

HowdoIappraise?

Mostlycommon

sense.

Youdonthavetobeastatisticalexpert!

Checklistshelp

you

focus

on

the

most

importantaspectsofthearticle.

Different

checklists

for

different

types

of

research.

Willhelpyoudecideifresearchisvalidand

relevant.


13/39

Researchmethods

Quantitative

Usesnumbersto

describeand

analyse

Usefulforfindingpreciseanswersto

definedquestions

Qualitative

Useswordstodescribe

andanalyse

Usefulforfindingdetailedinformation

aboutpeoples

perceptionsand

attitudes


14/39

Levelsofquantitativeevidence.

Systematicreviews.

Randomizedcontrolled

trials.

Prospectivestudies(cohortstudies).

Retrospective

studies

(case

control). Caseseriesandreports

Opinionsofrespectedauthorities.


15/39

SystematicReviews.

Thoroughsearchofliteraturecarried

out. AllRCTs (orotherstudies)onasimilarsubjectsynthesisedandsummarised. Metaanalysistocombinestatistical

findingsofsimilarstudies.


16/39

RandomisedControlledTrials

(RCTs)

Normaltreatment/placeboversusnewtreatment.

Participantsare

randomised.

Ifpossibleshouldbedoubleblinded.

Intentionto

treat

analysis


17/39

Cohort studies prospective

groups (cohorts) exposure to a risk factor

followed over a period of time compare rates of development of anoutcome of interest

Confounding factors and bias


18/39

Case control studies Retrospective

Subjects confirmed with a disease (cases)are compared with non-diseased subjects(controls) in relation to possible past

exposure to a risk factor.

Confounding factors and bias


19/39

Appraising original researchAre

the

results

valid?

Istheresearchquestionfocused?

Wasthe

method

appropriate?

Howwasitconducted?

Whataretheresults?

Howwasdatacollectedandanalysed?

Aretheysignificant?

Willthe

results

help

my

work

with

patients?


20/39

AppraisingRCTs

Recruitment and sample size Randomisation method and controls

Confounding factors Blinding

Follow-up Intention to treat analysis Censoring


21/39

Appraisingsystematicreviews.

Wasathoroughliteraturesearchcarriedout?

Publicationbias

paperswithmoreinteresting resultsaremorelikelytobe: Submitted

for

publication Acceptedforpublication

Publishedinamajorjournal

Publishedin

the

English

language


22/39

Publicationbias

AllSSRItrialsregisteredwithFDA

37studies

were

assessed

by

FDA

as

positive

36ofthesewerepublished. 22

studies

with

negative

or

inconclusive

resultswerenotpublishedand11were

writtenupaspositive.

Turneretal.NEJM,2008.


23/39

Reviewsingeneralmedicaljournals

50reviewsin4majorjournals19856

Nostatementofmethods

Summaryinappropriate

Currentsystematic

reviews

do

not

routinely

usescientificmethodstoidentify,assessandsynthesiseinformation (Mulrow,1987)


24/39

Istheresearchquestionfocused?

Patient (e.g.child) Intervention

(e.g.MMRvaccine)

Comparison (e.g.singlevaccines) Outcome (e.g.autism)


25/39

Areresultssignificant?

Howwasdatacollected?

Whichstatistical

analyses

were

used?

Howprecisearetheresults?

Howare

the

results

presented?


26/39

Intentiontotreatanalyses

Analysingpeople,attheendofthetrial,inthe

groupsto

which

they

were

randomised,

even

iftheydidnotreceivetheintended

intervention


27/39

Statisticalanalyses

Oddsratios,absoluteandrelativerisks/benefits,hazardratios/relativehazards

Thelikelihoodofsomethinghappeningvs thelikelihoodofsomethingnothappeningNumber

needed

to

treat

(NNT)

Thenumberofpeopleyouwouldneedtotreattoseeoneadditionaloccurrenceofaspecific

beneficial

outcome


28/39

OddsRatioDiagrams.(Blobbograms orForestPlots.)


29/39

OddsRatioDiagrams

Lineof

no

effect

no

difference

between

treatmentandcontrolgroup

Result(blob)

to

the

Left

of

the

line

of

no

effect

=Lessoftheoutcomeinthetreatmentgroup.

ResulttotheRightoftheline=Moreoftheoutcome.

BUT Istheoutcomegoodorbad?


30/39

Cardiacdeaths less=good


31/39

Smokingcessation more=good


32/39

ConfidenceIntervals.

Longerconfidenceinterval=less

confidentof

results

wider

range.

Shorterconfidenceinterval=more

confident narrower

range.

Crosseslineofnoeffect/nosignificance=

Inconclusiveresults.


33/39

Confidenceintervals


34/39

PValues.

Pstandsforprobability

howlikelyisthe

resultto

have

occurred

by

chance?

Pvalueoflessthan0.05meanslikelihoodofresultsbeingduetochanceislessthan1in20=statistically

significant.

Pvaluesandconfidenceintervalsshouldbe

consistent


35/39

NumberNeededtoTreat

Thenumberofpeopleyouwouldneedto

treatto

see

one

additional

occurrence

of

aspecificbeneficialoutcome.

The

number

of

patients

that

need

to

be

treatedtopreventonebadoutcome.


36/39

Areresultsrelevant?

CanIapplytheseresultstomyownpractice?

Ismylocalsettingsignificantlydifferent?

Arethesefindingsapplicabletomypatients?

Arefindingsspecific/detailedenoughtobe

applied? Werealloutcomesconsidered?


37/39

Thegoodnews!

Someresourceshavealreadybeen

critically

appraised

for

you. Anincreasingnumberofguidelinesand

summaries

of

appraised

evidence

are

availableontheinternet.


38/39

Summary.

Searchforresourcesthathavealreadybeen

appraisedfirst,

e.g.

Guidelines,

Cochrane

systematic

reviews.

Searchdownthroughlevelsofevidence,e.g.systematic

reviews,

RCTs.

Usecheckliststoappraiseresearch.

Howcan

these

results

be

put

into

practice?


39/39

Questions?

quantitative presentation

Documents