ebd for dental staff seminar 2: core critical appraisal dominic hurst [email protected]...
TRANSCRIPT
Structure of the seminars
Seminar 1• Recap of EBD• Using an educational prescription• Structured questions and search
Seminar 2• Critical appraisal
Seminar 3 • Communicating evidence to patients• Getting evidence into practice
Workshop• Focused work on one or more of these
Why critically appraise a study?Put rubbish in, get rubbish out
Critical appraisal is the process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in a particular context.**http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/What_is_critical_appraisal.pdf
Critical appraisal
Relevance
Value
Trust-worthiness
Three key questions
• Is the study valid?• Internal validity
• What are the results?• Size and precision of the results
• Can I use it with the patients I manage in my context?• External validity and usefulness in this setting
Is the study valid?Are the results what they claim to be?
Internal validity
• In intervention or risk factor questions:• The degree of confidence we have that the difference in outcomes between 2
or more groups is due to the intervention / exposure and not something else• Regular fluoride varnish reduces caries incidence in children by 30-40%• TMJD resolves of its own accord in 70% of patients after 3 months
Does a red pill make a mouse grow more than a blue pill?
What messes up internal validity?
• Confounder variables or factors• Any variable other than the one of interest, that can influence the outcome
Study concludes that “night shifts cause dental caries”. Is there any other explanation?
• Confounders lead to bias• A systematic deviation from the truth
• High internal validity if bias is low / not present• Low internal validity if bias is high
Using critical appraisal checklists
• RCTs
• Systematic reviews
What are the results?Think absolutes rather than relatives…
Size (or magnitude) of the effect
• In a trial with 2000 patients having non-surgical extractions• 1000 get the socket rinsed with chlorhexidine• 1000 get the socket rinsed with water• We follow up for 14 days• We count how many get dry socket
I C
No dry socket
950 900 1850
Dry socket
50 100 150
1000 1000 2000
• Risk of getting a dry socket with the intervention?• 50/1000=5%
• Risk of getting a dry socket with the comparison?• 100/1000=10%
• Absolute risk reduction (ARR)?• 10%-5%=5% (0.05)
• Relative risk reduction (RRR)?• (10%-5%)/10%=50%
• Number needed to treat (NNT) with the chlorhexidine rather than water to prevent one additional dry socket?• 1/ARR=1/0.05=20
I C
No dry socket
955 990 1985
Dry socket
5 10 15
1000 1000 2000
• Risk of getting a dry socket with the intervention?• 5/1000=0.5%
• Risk of getting a dry socket with the comparison?• 10/1000=1.0%
• Absolute risk reduction (ARR)? • 1.0%-0.5%=0.5% (0.005)
• Relative risk reduction (RRR)?• (1.0%-0.5%)/1.0%=50%
• Number needed to treat (NNT) with the chlorhexidine rather than water to prevent one additional dry socket?• 1/ARR=1/0.005=200
Other ways results are expressed
• Mean difference• E.g. mean difference in probing depth
• Odds ratios• Odds: the chance that something happens / the chance that it doesn’t
I C
No dry socket
950 900 1850
Dry socket
50 100 150
1000 1000 2000
• Odds of a dry socket if using Intervention?• 50/950=0.053
• Odds of a dry socket if using comparison?• 100/900=0.111
• Odds ratio?• 0.053/0.11=0.477
Results in Systematic Reviews: Forrest Plot
Precision and statistical significance
• 95% Confidence intervals• We are 95% confident that the population result would lie within this range
either side of the study result
• P values• If P<0.05 = a less than 1/20 chance that the result is due to chance
• P<0.005 means there is a less than 1 in 200 chance the result is due to chance
Can you use the results locally?External validity, your skills, your patients and your resources
Things to consider of any study
• Are all the outcomes that would be of interest to you reported?• Think of harms as well as benefits
• Are the patients that were in the study so different?• Is what is being done feasible by a student / you in this setting?
Key messages
• Critical appraisal asks 3 questions:• Is the study valid?• What are the results?• Can they be used locally?
• Internal validity affected by confounding factors that cause bias• Relative outcomes can be misleading…so opt for absolutes if you can
Thanks for participatingDominic [email protected]