evidence based medicine for the athletic trainer: what is it? july 11, 2015 boc approved ebp ceus...

Evidence Based Medicine for the Athletic Trainer: What is It?

July 11, 2015

BOC Approved EBP CEUs

Evidence-Informed Clinical Practice for the Athletic Trainer:

…while others wandered around…and dined with the rich and powerful …we on the other hand labored diligently…we first learned all the things that were creditably discovered by the ancients; then through deeds we both tested and practiced them.

-Galen (129-217 CE [disputed])Method of Medicine 9.4, 10.609K

Jennifer M. Medina McKeon, PhD, ATC, CSCSjmckeon@ithaca.edu

…while others wandered around…and dined with the rich and powerful …we on the other hand labored diligently…we first learned all the things that were creditably discovered by the ancients; then through deeds we both tested and practiced them.

-Galen (129-217 CE [disputed])Method of Medicine 9.4, 10.609K

“EBP is just a fad.”

What EBP is not…

“All this EBP stuff just means that they think that clinicians are all supposed to become researchers now”

We all know that you can prove anything with statistics…

…I recently proved that nobody likes statistics, except a few professors

- Chottiner (1990)

“EBP means that we all need to be doing stats on everything.”

What EBP is not…

What EBP is not…

“EBP means that someone should read all the research that’s out there on a topic, and tell me what to do.”

What EBP is not…

“Ideally, we would like to see a clinician encounter an injury, and right there, pull out a laptop or tablet, and look up the most recent article on that injury, and incorporate what the study says right there on the spot.”

What is the EBP?

Evidence-Based Practice

Patient Values

Clinical Experience

Best Available Evidence

“Integration of the best available research evidence with clinical experience

and patient values to make clinical decisions” – Sackett et al, BMJ 1996.

Hierarchy of Clinical Research Evidence

Meta-analysesSystematic Reviews

Randomized Clinical Trials

Cohort Studies(Prospective)Case-Control Studies

(Retrospective)Case SeriesCase StudiesUnpublished Clinical Observations

More Control

Less Risk of Bias

More Subjects Involved B

ET

TE

R R

ES

EA

RC

H E

VID

EN

CE

HA

RD

ER

TO

DO

Critical Appraisal

Critical Appraisal

Critical Appraisal

More than Peer Review

Vital Step to making INFORMED clinical decisions

Combination of External Evidence (the stuff you find) & the Internal Evidence (the stuff you know)

Critical Appraisal

More than Peer Review

Vital Step to making INFORMED clinical decisions

Combination of External Evidence (the stuff you find) & the Internal Evidence (the stuff you know)

Critical Appraisal

More than Peer Review

Vital Step to making INFORMED clinical decisions

Combination of External Evidence (the stuff you find) & the Internal Evidence (the stuff you know)

Critical Appraisal

More than Peer Review

Vital Step to making INFORMED clinical decisions

Combination of External Evidence (the stuff you find) & the Internal Evidence (the stuff you know)

What is the BEST SPECIAL TEST???What is the BEST TREATMENT???

What makes a test good?Accurate (Valid)Consistent results (Reliable)

YUCK – that sounds like research

RESEARCH EVIDENCE

What makes a test good?Accurate (Valid)Consistent results (Reliable)

YUCK – that sounds like research

RESEARCH EVIDENCE: The Best Tests

What makes a test good? Accurate (Valid)Consistent results (Reliable)

RESEARCH EVIDENCE: The Best Tests

SENSITIVITY SPECIFICTY

Definition Proportion of patients WITH the injury who test POSITIVE

Proportion of patients WITHOUT the injury who test NEGATIVE

100% Sensitivity (Perfect Score)

Test correctly identifies every person who HAS the injury

Test correctly identifies every person who DOES NOT HAVE the injury

Outcome True Positive Rate True Negative Rate

“Ideal” Test Result Negative Positive

“Ideal” Test Interpretation(What to tell the Patient)

Definitely NEGATIVE(Patient is OK)

Definitely POSITIVE(Patient is Injured)

“The RULE” RULE OUT (SnOUT)If Test is Negative, You can Rule Out injury (Patient is OK)

RULE IN (SpIN)If Test is Positive, You can Rule In injury (Patient is Injured)

Poor Test Lots of FALSE NEGATIVES Lots of FALSE POSITIVES

Adapted from http://www.aaronswansonpt.com/sensitivity-and-specificity/Accessed January 23, 2015

What makes a test good? Accurate (Valid)Consistent results (Reliable)

RESEARCH EVIDENCE: The Best Tests

Adapted from http://www.aaronswansonpt.com/sensitivity-and-specificity/Accessed January 23, 2015

SENSITIVITY SPECIFICTY

Definition Proportion of patients WITH the injury who test POSITIVE

Proportion of patients WITHOUT the injury who test NEGATIVE

100% Sensitivity (Perfect Score)

Test correctly identifies every person who HAS the injury

Test correctly identifies every person who DOES NOT HAVE the injury

Outcome True Positive Rate True Negative Rate

“Ideal” Test Result Negative Positive

“Ideal” Test Interpretation(What to tell the Patient)

Definitely NEGATIVE(Patient is OK)

Definitely POSITIVE(Patient is Injured)

“The RULE” RULE OUT (SnOUT)If Test is Negative, You can Rule Out injury (Patient is OK)

RULE IN (SpIN)If Test is Positive, You can Rule In injury (Patient is Injured)

Poor Test Lots of FALSE NEGATIVES Lots of FALSE POSITIVES

What makes a test good? Accurate (Valid)Consistent results (Reliable)

RESEARCH EVIDENCE: The Best Tests

Adapted from http://www.aaronswansonpt.com/sensitivity-and-specificity/Accessed January 23, 2015

SENSITIVITY SPECIFICTY

Definition Proportion of patients WITH the injury who test POSITIVE

Proportion of patients WITHOUT the injury who test NEGATIVE

100% Sensitivity (Perfect Score)

Test correctly identifies every person who HAS the injury

Test correctly identifies every person who DOES NOT HAVE the injury

Outcome True Positive Rate True Negative Rate

“Ideal” Test Result Negative Positive

“Ideal” Test Interpretation(What to tell the Patient)

Definitely NEGATIVE(Patient is OK)

Definitely POSITIVE(Patient is Injured)

“The RULE” RULE OUT (SnOUT)If Test is Negative, You can Rule Out injury (Patient is OK)

RULE IN (SpIN)If Test is Positive, You can Rule In injury (Patient is Injured)

Poor Test Lots of FALSE NEGATIVES Lots of FALSE POSITIVES

“New Rule” High SNNNHigh SeNsitivity, Negative test, Negative pt.

High SPPPHigh SPecificity, Positive test, Positive pt.

BiasComparison

Generalizability

RESEARCH EVIDENCE: The Best TestsQUESTIONS TO ASK

Generalizability

Is the Test Relevant to your practice?

Who was the Test performed on?BiasComparison

QUESTIONS TO ASK

RESEARCH EVIDENCE: The Best Tests

BiasComparison

Generalizability

RESEARCH EVIDENCE: The Best Tests

Who did the Test?Was the Test compared to a Gold Standard?Was the Test tried in a group of Patients who could conceivably have the Injury?

QUESTIONS TO ASK

BiasComparison

Generalizability

RESEARCH EVIDENCE: The Best Tests

BiasComparison

Generalizability

RESEARCH EVIDENCE: The Best Tests

BiasComparison

Generalizability

Did the testers “know” if the patient had the Injury?Did ALL patients tested also get tested with the Gold Standard?

QUESTIONS TO ASK

RESEARCH EVIDENCE: The Best Tests

RESEARCH EVIDENCE: The Best Tests

BiasComparison

Generalizability1. Is the Test Relevant to your practice?

2. Who was the Test performed on?

3. Who did the Test?

4. Was the Test compared to a Gold Standard?

5. Was the Test tried in a group of Patients who could conceivably have the Injury?

6. Did the testers “know” if the patient had the Injury?

7. Did ALL patients tested also get tested with the Gold Standard?

QUESTIONS TO ASK

Improve your ability to Critically Analyze Diagnostic Test research

IMPROVE YOUR OWN PRACTICE

=

PREDICTION

BEST PREDICTORS

Get rid of these…they don’t add much

Clinical Prediction Rules (the other CPR)

Combination of 3 or more specific patient characteristics that provides a quantifiable probability for diagnosis (or prognosis or benefit for a procedure)

Best combination of S/S to predict probability of Injury

Calculated for Maximized Prediction

Support the decisions that you make

Considerations

Must be validated

Do not guarantee n outcome (a probability)

Used fore making predictions, not recommendations (what is likely to happen, not what to do next)

“Suggest” vs. “Recommend” next course of action Clinical Decision Rule

Application, not Development

Clinical Prediction Rules (the other CPR)

Combination of 3 or more specific patient characteristics that provides a quantifiable probability for diagnosis (or prognosis or benefit for a procedure)

Best combination of S/S to predict probability of Injury

Calculated for Maximized Prediction

Support the decisions that you make

Considerations

Must be validated

Do not guarantee an outcome (they’re a probability)

Used fore making predictions, not recommendations (what is likely to happen, not what to do next)

“Suggest” vs. “Recommend” next course of action (vs. Clinical Decision Rule)

Application, not Development

RESEARCH EVIDENCE: The Best Treatments

What makes a treatment good?

RESEARCH EVIDENCE: The Best Treatments

Posing a Clinical Question

RESEARCH EVIDENCE: The Best Treatments

Posing a Clinical Question

P

I

C

O

RESEARCH EVIDENCE: The Best Treatments

Posing a Clinical Question

P Patient or Problem or PopulationWhat are the most important Characteristics of your Patient?

I

C

O

RESEARCH EVIDENCE: The Best Treatments

Posing a Clinical Question

P Patient or Problem or PopulationWhat are the most important Characteristics of your Patient?

IIndependent Variable(Intervention or Assessment or Exposure)

What is the main Intervention, Prognostic factor, Diagnostic factor, or Exposure you are considering for your patient?

C

O

RESEARCH EVIDENCE: The Best Treatments

Posing a Clinical Question

P Patient or Problem or PopulationWhat are the most important Characteristics of your Patient?

IIndependent Variable(Intervention or Assessment or Exposure)

What is the main Intervention, Prognostic factor, Diagnostic factor, or Exposure you are considering for your patient?

C Comparison or ControlWhat is the main alternative to the Intervention / Assessment / Exposure?

O

RESEARCH EVIDENCE: The Best Treatments

Posing a Clinical Question

P Patient or Problem or PopulationWhat are the most important Characteristics of your Patient?

IIndependent Variable(Intervention or Assessment or Exposure)

What is the main Intervention, Prognostic factor, Diagnostic factor, or Exposure you are considering for your patient?

C Comparison or ControlWhat is the main alternative to the Intervention / Assessment / Exposure?

O Outcome of Interest What is considered “success”

RESEARCH EVIDENCE: The Best Treatments

Posing a Clinical Question

P Patient or Problem or PopulationWhat are the most important Characteristics of your Patient?

IIndependent Variable(Intervention or Assessment or Exposure)

What is the main Intervention, Prognostic factor, Diagnostic factor, or Exposure you are considering for your patient?

C Comparison or ControlWhat is the main alternative to the Intervention / Assessment / Exposure?

O Outcome of Interest What is considered “success”

RESEARCH EVIDENCE: The Best Treatments

QUESTIONS TO ASK

The PEDro Scale http://www.pedro.org.au/english/downloads/pedro-scale/Accessed January 23, 2015

RESEARCH EVIDENCE: The Best Treatments

BiasComparison

Generalizability

RESEARCH EVIDENCE: The Best Treatments

QUESTIONS TO ASK

BiasComparison

Generalizability

Who was the Treatment performed on?

Did all of the patients present similarly?

RESEARCH EVIDENCE: The Best Treatments

BiasComparison

Generalizability

RESEARCH EVIDENCE: The Best Treatments

BiasComparison

Generalizability

QUESTIONS TO ASK

(In other words, were the patients who received the treatment Really better than the controls?)

Did they compare the Groups?

Were they meaningfully different?

RESEARCH EVIDENCE: The Best Treatments

BiasComparison

Generalizability

RESEARCH EVIDENCE: The Best Treatments

BiasComparison

Generalizability

QUESTIONS TO ASK

Did the individuals involved already have an “idea” of what should happen?

In other words: No Blinding of - Patients- Therapists- Investigators

RESEARCH EVIDENCE: The Best Treatments

BiasComparison

Generalizability

RESEARCH EVIDENCE: The Best Treatments

BiasComparison

Generalizability1. Who was the Treatment performed on?

2. Who Did all the patients present similarly?

3. Did they compare the groups?

4. Where they meaningfully different?

5. Did the individuals involved have an “idea” of what should happen

QUESTIONS TO ASK

Improve your ability to Critically Analyze Intervention research

IMPROVE YOUR OWN PRACTICE

RESEARCH EVIDENCE: Summary

Specifically, we covered:

• Sackett’s 3-pronged EBP model

• Levels of Evidence

• Critical appraisal - Diagnosis (Diagnostic accuracy & CPRs)

• Critical appraisal - Intervention (PICO, Therapeutic studies)

We focused on Foundational concepts of EBP

Evidence-Based Practice

Patient Values

Clinical Experience

Best Available Evidence

“Integration of the best available research evidence with clinical experience

and patient values to make clinical decisions” – Sackett et al, BMJ 1996.

Read not to

contradict and

confute, nor to

believe and take

for granted…but

to weigh and

consider

- Francis Bacon (1561-

1626)

“Commitment to Excellence”

jmckeon@ithaca.edu