Evidence-Based Practice in Clinical Psychology:

What It Is, Why It Matters, What You Need to Know

Bonnie Spring, Ph.D., ABPPBonnie Spring, Ph.D., ABPPNorthwestern UniversityNorthwestern University

Why it matters: EBBP Why it matters: EBBP RationaleRationale

improve quality and accountability for health care practice (IOM, 2001, Crossing the Quality Chasm)

shared vocabulary and concepts for transdisciplinary, biopsychosocial research, practice, health care policy

stimulate development of evidence base for behavioral treatments

Why it matters: Why it matters: Potentionally Useful InfrastructurePotentionally Useful Infrastructure

Clinical Practice Guidelines:Clinical Practice Guidelines: • Increasingly based on ongoing systematic review of research

(esp. RCTs) (e.g., USPTF, Cochrane, CDC/AHRQ)• Research reporting guidelines (CONSORT, TREND, QUOROM)• Evidence grading & knowledge synthesis systems (e.g.,

GRADE, AHRQ)• Policy, often coverage/reimbursement implications (VA/DOD,

CMS, NICE) (P4P?)

Evidence-Based Practice: Evidence-Based Practice: (life-long learning)• Question formulation, search strategies, critical appraisal• SUMSEARCH• Clinical Evidence, First Consult, BMJ updates, Best Evidence

Topics, CATCRAWLER, CATBANK – clinical scenario & bottom line

OverviewOverview History of evidence-based practice History of evidence-based practice

(EBP)(EBP) Core elements of EBPCore elements of EBP EBP pedagogy in psychologyEBP pedagogy in psychology EBP pedagogy in other health EBP pedagogy in other health

disciplinesdisciplines Useful infrastructure and potential Useful infrastructure and potential

opportunities for synergyopportunities for synergy

Origins of Evidence-Based Practice

Emergence of Emergence of Evidence-Based Evidence-Based

MedicineMedicine1910 - Flexner report1910 - Flexner report : :155(31!)155(31!) 96 (1915) 76(1930) 96 (1915) 76(1930)1972 - Archie Cochrane – epidemiology, health services research - 1972 - Archie Cochrane – epidemiology, health services research -

Effectiveness and Efficiency: Random Reflections on Health Effectiveness and Efficiency: Random Reflections on Health ServicesServices

1973 – John Wennberg – widespread practice variation1973 – John Wennberg – widespread practice variation1982 - clinical epidemiology determinants and consequences of 1982 - clinical epidemiology determinants and consequences of

health care decisions (McMaster U – David Sackett, Gordon health care decisions (McMaster U – David Sackett, Gordon Guyatt)Guyatt)

1985 – IOM: 15% medical practices evidence-based [2001 1985 – IOM: 15% medical practices evidence-based [2001 Crossing Crossing the Quality Chasmthe Quality Chasm]]

1990 - Evidence-based medicine, Brian Haynes & Ann McKibbon – 1990 - Evidence-based medicine, Brian Haynes & Ann McKibbon – search strategiessearch strategies

1992-3 -Cochrane Collaboration1992-3 -Cochrane Collaboration

2000 - 2000 - Sackett - How to Practice and Teach EBMHow to Practice and Teach EBM

What do we mean by What do we mean by “evidence-based practice?”“evidence-based practice?”

Alternative Definitions of Evidence-Based Practice

Guidelines:Guidelines: (public health, medicine)public health, medicine) – – focus on problem/disorder & level of evidence for practices (based on systematic review) (e.g., NICE, VA, apa)

• ESTs:ESTs: (psychology)(psychology) focus on intervention (& disorder)

EBP:EBP: (psychology, medicine, nursing, social work) focus on decision-making about individual patients

Idiographic……….. Lifelong Learning

Nomothetic

APA Policy Statement APA Policy Statement adopted August 2005adopted August 2005

““Evidence-based practice in psychology is Evidence-based practice in psychology is the the integrationintegration of the best available of the best available researchresearch with with clinical expertiseclinical expertise in the in the context of context of patientpatient characteristics, culture, characteristics, culture, and and preferencespreferences.”.”

-adapted from IOM, 2001 & Sackett, 2000

Best available research evidence

Patient’s values, characteristics, and circumstances

Clinical Expertise

Clinical Decision-MakingClinical Decision-Making

Syllabus ProjectSyllabus Project

PromptPrompt: Does anyone on the list teach a : Does anyone on the list teach a course on evidence-based practice (EBP)?  course on evidence-based practice (EBP)?  Specifically, I am searching for syllabi that Specifically, I am searching for syllabi that cover one or more "legs" of the three-cover one or more "legs" of the three-legged EBP stool:  a) research evidence, b) legged EBP stool:  a) research evidence, b) clinical expertise, c) patient values, clinical expertise, c) patient values, preferences, characteristics. preferences, characteristics. 

November, 2006

Listservs SampledListservs Sampled ABCTABCT APA Division 12APA Division 12 SSCPNET (Section III, Div 12)SSCPNET (Section III, Div 12) CUDCPCUDCP APA Division 38APA Division 38 ABMRABMR SBM EBBM, MRBC, Obesity, CA SIGsSBM EBBM, MRBC, Obesity, CA SIGs

OutcomeOutcome 39 syllabi 39 syllabi 17 additional recommended articles 17 additional recommended articles

and booksand books 273 page document273 page document Discipline:Discipline:

30 psychology30 psychology 3 public health3 public health 3 medicine3 medicine 1 nursing 1 nursing 1 PE/health/sport studies1 PE/health/sport studies

140 requests140 requestsNovember, 2006

Evidence-Based PracticeEvidence-Based Practice

ModalModal

Course TitleCourse Title: CBT, EST, : CBT, EST, EVT, Psychological EVT, Psychological Interventions, Interventions, Psychotherapy ResearchPsychotherapy Research

TextsTexts:: Barlow, Barlow, Handbook Handbook Psychologic DisordersPsychologic Disorders, , Bergen & GarfieldBergen & Garfield Handbook of Handbook of Psychotherapy and Psychotherapy and Behavior ChangeBehavior Change

Content:Content: ESTsESTs

AdditionalAdditional

Additional TextsAdditional Texts::-Persons, -Persons, Case Case ConceptualizationConceptualization--Dawes, Dawes, House of CardsHouse of Cards

Additional ContentAdditional Content::-Assessment-Assessment-Case formulation, -Case formulation, functional analysisfunctional analysis-Clinical judgment-Clinical judgment-Diversity-Diversity-Iatrogenic effects-Iatrogenic effects-Research methods-Research methods

Content Distribution of Books on EBP

0

20

40

60

80

100

120

EBM (Sackettet.al)

EBN (Craig,Smyth, et.al)

SW (Gibbs) LIB (McKibbon) PSYCH(Norcross,

Beutler, Levant)

What is EBP?How to do EBPHow to teach EBP

courtesy of Barbara Walker, Indiana University, 2006

Best available research evidence

Patient’s values, characteristics, and

circumstances

Clinical Expertise

Clinical Decision-MakingClinical Decision-Making

ResearcherDesign

Conduct Analysis

Reporting

SynthesizerLocateCritically appraiseMeta-analysis

ConsumerLocateAppraise quality & relevanceIntegrate

ClinicianCommunicateAssess patient

Deliver EBPPatientUnderstandingPreferencesAccess

Researcher Training in Researcher Training in Psychology versus MedicinePsychology versus Medicine

PsychologyPsychology• DesignDesign

Correlational Correlational (convenience classes)(convenience classes)

Experimental (from Experimental (from animal studies)animal studies)

• ConductConduct Brief, tight controlBrief, tight control Little missing data; Little missing data;

replace casesreplace cases• Analysis - Analysis - completercompleter• ReportingReporting

Clinical MedicineClinical Medicine• DesignDesign

Observational Observational (population)(population)

Clinical Trial –Clinical Trial –test of policy applied to population

• Conduct Long, intercurrent events Missing data;

• Analysis – ITT• Reporting – CONSORT

Researcher, Synthesizer, Researcher, Synthesizer, Consumer Training in AnalysisConsumer Training in Analysis

PsychologyPsychology• ANOVA/regressionANOVA/regression

Clinical MedicineClinical Medicine• Odds RatiosOdds Ratios

Epidemiology Terminology

Absolute risk (p[disease] in a particular population)Absolute risk (p[disease] in a particular population)

Relative risk (p[disease/exposed]/p[disease/unexposed)Relative risk (p[disease/exposed]/p[disease/unexposed)

Attributable risk (p[disease/exposed] -p[disease/unexposed)Attributable risk (p[disease/exposed] -p[disease/unexposed)

Number needed to harm (1/attributable risk)Number needed to harm (1/attributable risk)

Odds ratio Odds ratio (odds[disease/exposed]/odds[disease/unexposed])(odds[disease/exposed]/odds[disease/unexposed])

Clinical SignificanceClinical Significance NNHNNH = 5. = 5. If 5 patients treated with If 5 patients treated with

TX1, 1 would be more likely to have TX1, 1 would be more likely to have AE than if all had received TX0AE than if all had received TX0

NNTNNT = 13. = 13. 13 patients would need 13 patients would need to be treated with TX1 to see one to be treated with TX1 to see one success not seen with TX0success not seen with TX0

Reporting: Consort Flow Reporting: Consort Flow DiagramDiagram

Consolidated Consolidated Standards of Standards of Reporting Trials Reporting Trials (CONSORT)(CONSORT)www.consort-statement.org

Excerpt from CONSORT checklistExcerpt from CONSORT checklistMETHODS

Participants3 Eligibility criteria for participants and the

settings and locations where the data were collected.Interventions 4 Precise details of the interventions intended for each group and how

and when they were actually administered.Objectives 5 Specific objectives and hypotheses.

Outcomes 6 Clearly defined primary and secondary outcome measures and, when applicable, any methods used to enhance the quality of measurements (e.g., multiple observations, training of assessors).

Sample size 7 How sample size was determined and, when applicable, explanation of any interim analyses and stopping rules.

Randomization --Sequence generation

8 Method used to generate the random allocation sequence, including details of any restrictions

(e.g., blocking, stratification)Randomization --

Allocation concealment

9 Method used to implement the random allocation sequence (e.g., numbered containers or central telephone), clarifying whether the sequence was concealed until interventions were assigned.

Randomization --Implementation

10 Who generated the allocation sequence, who enrolled participants, and who assigned participants to their groups.

Blinding (masking) 11 Whether or not participants, those administering the interventions, and those assessing the outcomes were blinded to group assignment.

When relevant, how the success of blinding was evaluated.

Evidence Synthesizer Evidence Synthesizer and Consumer Skillsand Consumer Skills

Best available research evidence

Patient’s values, characteristics, and

circumstances

Clinical Expertise

Clinical Decision-MakingClinical Decision-Making

ResearcherDesign

Conduct Analysis

Reporting

SynthesizerLocateCritically appraiseMeta-analysis

Evidence User LocateAppraise quality & relevanceIntegrate

ClinicianCommunicateAssess patient

Deliver EBPPatientUnderstandingPreferencesAccess

SynthesizerSynthesizer: : Systematic Reviewer- Systematic Reviewer- explicit, systematic, transparent to avoid bias

Specific research question (PICO)Specific research question (PICO) Search protocol to select papers – key wordsSearch protocol to select papers – key words

systematic search of the literature (systematic search of the literature (EMBASE, CINAHLEMBASE, CINAHL, , Cochrane Controlled Trial register, DARE)Cochrane Controlled Trial register, DARE)

explicit inclusion and exclusion criteriaexplicit inclusion and exclusion criteria Explicit, transparent rating of methodological Explicit, transparent rating of methodological

qualityquality Data extractionData extraction Analysis: qualitative or quantitativeAnalysis: qualitative or quantitative ConclusionConclusion Discussion of strengths and limitationsDiscussion of strengths and limitations

The 5 Step EBM Model for The 5 Step EBM Model for Evidence Users Evidence Users (Consumers) (Consumers)

Ask: formulate the question formulate the question Acquire:Acquire: evidence - search for evidence - search for

answersanswers Appraise: the evidence for the evidence for

quality and relevancequality and relevance Apply the results the results Assess the outcome the outcome

AskAsking: Well-Built Clinical ing: Well-Built Clinical QuestionsQuestions

BackgroundBackground:: What are effective What are effective treatments for bulimia nervosa?treatments for bulimia nervosa?

ForegroundForeground: : In patients with In patients with PPatient: : binge eating disorderbinge eating disorderIntervention: does interpersonal

therapyComparison: compared to CBT reduceOutcome: frequency of binge episodes

Critically appraising the evidence

Use of standardized a priori appraisal Use of standardized a priori appraisal methods to answer:methods to answer:

Is the evidence Is the evidence validvalid??• Internal validity Internal validity

Is the evidence Is the evidence applicable/relevantapplicable/relevant??• External validityExternal validity

Is the evidence Is the evidence clinically significantclinically significant??

Clinical Decision-MakingClinical Decision-Making Clinical epidemiology disciplineClinical epidemiology discipline

• study of determinants and study of determinants and consequences of clinical decisionsconsequences of clinical decisions

• apply EBP/5A’s/critical appraisal at apply EBP/5A’s/critical appraisal at clinical encounter to overcome clinical encounter to overcome automatic, unconscious decision-making automatic, unconscious decision-making biases biases (aka bad clinical intuition)(aka bad clinical intuition)

barriers between research and practicebarriers between research and practice

30 kg of guidelines per family doctor per year

25000 biomedical journals in print

8000 articles published per day

95% of studies cannot reliably guide clinical decisions

2001 Bazian Ltd

Clinical Decision-MakingClinical Decision-Making Health InformaticsHealth Informatics disciplinediscipline

infrastructure, resources, devices, infrastructure, resources, devices, structures (e.g., algorithms, structures (e.g., algorithms, guidelines) needed to store, retrieve, guidelines) needed to store, retrieve, manage and use health information manage and use health information and the time and place that a and the time and place that a decision needs to be made. decision needs to be made. -Decision support. -Decision support.

Secondary Synthesized Secondary Synthesized EvidenceEvidence

(AKA “evidence-based capitulation”)(AKA “evidence-based capitulation”) Research proliferates rapidly. Clinical Research proliferates rapidly. Clinical

performance demands increase. Practicing performance demands increase. Practicing clinicians too busy to use all EBM steps will all clinicians too busy to use all EBM steps will all patients. patients.

Increased focus on pithy clinical practice Increased focus on pithy clinical practice guidelines, synopses, and structured abstractsguidelines, synopses, and structured abstracts• MD ConsultMD Consult• ACP Journal ClubACP Journal Club• Cochrane Database of Systematic ReviewsCochrane Database of Systematic Reviews• ““Up-to-date”Up-to-date”• InfoPOEMS (Patient Oriented Evidence that Matters)InfoPOEMS (Patient Oriented Evidence that Matters)

Best available research evidence

Patient’s values, characteristics, and

circumstances

Clinical Expertise

Clinical Decision-MakingClinical Decision-Making

ResearcherDesign

Conduct Analysis

Reporting

SynthesizerLocateCritically appraiseMeta-analysis

ConsumerLocateAppraise quality & relevanceIntegrate

ClinicianCommunicateAssess patient

Deliver EBPPatientUnderstandingPreferencesAccess

Clinically Supervised Training Clinically Supervised Training in Evidence-Based Treatmentin Evidence-Based Treatment

Needs work: 2005-2006 papers by Needs work: 2005-2006 papers by Woody and by Weissman Woody and by Weissman

Best available research evidence

Patient’s values, characteristics, and

circumstances

Clinical Expertise

Clinical Decision-MakingClinical Decision-Making

ResearcherDesign

Conduct Analysis

Reporting

SynthesizerLocateCritically appraiseMeta-analysis

ConsumerLocateAppraise quality & relevanceIntegrate

ClinicianCommunicateAssess patient

Deliver EBPPatientUnderstandingPreferencesAccess

Patient PreferencesPatient Preferences Shared decision-making requires information only available to requires information only available to

patient (e.g., valuation of harms/hassles, alternative outcomes & patient (e.g., valuation of harms/hassles, alternative outcomes & treatments)treatments)

Utility assessment: All possible outcomes assigned a value between 0 : All possible outcomes assigned a value between 0 (death) and 1 (perfect health).(death) and 1 (perfect health).

Time trade-off approachTime trade-off approach• The proportion of life in a particular health state (e.g., severe The proportion of life in a particular health state (e.g., severe

depression) that you would give up to attain perfect health (e.g., 30%). depression) that you would give up to attain perfect health (e.g., 30%). Utility of that health state is 1-(30%) = .70Utility of that health state is 1-(30%) = .70

Standard gamble approachStandard gamble approach• The point where you are indifferent to the choice between spending the The point where you are indifferent to the choice between spending the

rest of your life in the health state in question and a gamble between rest of your life in the health state in question and a gamble between perfect health and instant death where the probability of perfect health perfect health and instant death where the probability of perfect health represents the utility of the health state.represents the utility of the health state.

Teaching evidence-based Teaching evidence-based practice = teaching a practice = teaching a

processprocess DidacticsDidactics Small groups, problem-based learning Small groups, problem-based learning Preceptorships/clinical supervisionPreceptorships/clinical supervision Standardized patients and evidence Standardized patients and evidence

stationsstations Embedded throughout curriculum Embedded throughout curriculum

Medical Decision Making in the NU-Medical Decision Making in the NU-FSM curriculumFSM curriculum

MDM-I (first week of medical school)MDM-I (first week of medical school)• Sensitivity, specificity, pre- and post-test probabilities, Sensitivity, specificity, pre- and post-test probabilities,

innumeracy, uncertainty in medicineinnumeracy, uncertainty in medicine MDM-II (last two weeks of M1 year)MDM-II (last two weeks of M1 year)

• EpidemiologyEpidemiology• StatisticsStatistics

MDM-III (beginning of M2 Spring Quarter)MDM-III (beginning of M2 Spring Quarter)• Decision analysisDecision analysis• Meta-analysisMeta-analysis• Cost-effectiveness analysisCost-effectiveness analysis• Clinical guidelinesClinical guidelines

M3 MDM (once a month in M3 year)M3 MDM (once a month in M3 year)• Review papers pertaining to clinical casesReview papers pertaining to clinical cases• Use of CATUse of CAT

Evidence-Based Behavioral Practice (EBBP)Evidence-Based Behavioral Practice (EBBP)

NIH Office of Behavioral and Social Sciences NIH Office of Behavioral and Social Sciences Research contract N01-LM-6-3512: Research contract N01-LM-6-3512: Resources for Training in Evidence-Resources for Training in Evidence-

Based Behavioral Practice, Based Behavioral Practice, 2006 - 20112006 - 2011

OBSSR 5-Year PlanOBSSR 5-Year Plan Year 1Year 1: : developdevelop training websitetraining website, , Council, Council,

Scientific Advisory BoardScientific Advisory Board, , white paperwhite paper on training, on training, skills, competencies reflecting education in skills, competencies reflecting education in evidence-based behavioral practice (EBBP)evidence-based behavioral practice (EBBP)

Year 2Year 2: d: develop, implement a web-based, evelop, implement a web-based, research-focused training module(s)research-focused training module(s) on EBBP; on EBBP; field test in graduate curriculafield test in graduate curricula

Year 3Year 3: l: launch interactive web-based training aunch interactive web-based training courses; establish courses; establish practice networkpractice network, develop first , develop first EBBP EBBP clinical practice training moduleclinical practice training module

OBSSR 5-Year PlanOBSSR 5-Year Plan Year 4Year 4: With practice network, develop : With practice network, develop modulesmodules on on

application of evidence-based application of evidence-based clinical decision-clinical decision-makingmaking to intervention with specific cases to intervention with specific cases. . Field Field test in test in internship/residency/post-doctoral training internship/residency/post-doctoral training programs and practice network.programs and practice network.

Year 5Year 5: Link website to systematic reviews of : Link website to systematic reviews of behavioral interventions, treatment manuals, behavioral interventions, treatment manuals, outcome assessments. Develop and field test clinical outcome assessments. Develop and field test clinical decision-making modules that decision-making modules that integrate patient integrate patient preference and clinical competency assessmentspreference and clinical competency assessments..

SuggestionsSuggestions To enhance the evidence base for psychological To enhance the evidence base for psychological

treatments and support lifelong learning, clinical treatments and support lifelong learning, clinical psychology training might benefit from enhanced psychology training might benefit from enhanced coverage of: coverage of:

• Researcher skills in methods: clinical trial Researcher skills in methods: clinical trial design, analysis, reporting, synthesisdesign, analysis, reporting, synthesis

• Clinician training in 5-step (5A’s) EBP model – Clinician training in 5-step (5A’s) EBP model – cover 2 A’scover 2 A’s

Suggestions Suggestions • Psychology informatics could use infrastructure Psychology informatics could use infrastructure

development (PSYCinfo & Cochrane; development (PSYCinfo & Cochrane; library access; coverage in secondary synthesized ; coverage in secondary synthesized sources like Up-to-Date; practice-based research sources like Up-to-Date; practice-based research networks)networks)

• Psychology could use appropriate patient Psychology could use appropriate patient preference measures that support shared preference measures that support shared decision-makingdecision-making

• A discipline of clinical psychology decision-making A discipline of clinical psychology decision-making needs to develop to systematize integration of needs to develop to systematize integration of research evidence, clinical expertise, and patient research evidence, clinical expertise, and patient clinical data and preferencesclinical data and preferences

Concluding QuestionsConcluding Questions What training modules and materials What training modules and materials

would be helpful?would be helpful?

Will you partner with us to help Will you partner with us to help develop and try these out? develop and try these out?

The Evidence Pyramid forThe Evidence Pyramid forTreatment Effectiveness QuestionsTreatment Effectiveness Questions

***USE THE BEST EVIDENCE ***USE THE BEST EVIDENCE AVAILABLE***AVAILABLE***

Alternatives to evidence-based Alternatives to evidence-based medicinemedicine

Eminence based medicineEminence based medicine Eloquence based medicineEloquence based medicine Vehemence based medicineVehemence based medicine Nervousness based medicineNervousness based medicine

(Isaacs and Fitzgerald, 1999, BMJ)

Levels of Clinical Evidence in the Primary Levels of Clinical Evidence in the Primary Literature Literature (psycINFO, MEDLINE)(psycINFO, MEDLINE)

RCT, Practice Guideline RCT, Practice Guideline Consensus Development Consensus Development ConferenceConference

RandomizedRandomizedControlled TrialControlled Trial

Quality Quality ImprovementImprovement

Cohort Studies, RiskCohort Studies, RiskCohort StudiesCohort StudiesEtiologyEtiology

Cohort Studies, Prognosis, Cohort Studies, Prognosis, Survival AnalysisSurvival Analysis

Cohort Studies, Cohort Studies, Case Control, Case Control, Case SeriesCase Series

PrognosisPrognosis

Randomized Controlled Randomized Controlled Trial, Double Blind, Clinical Trial, Double Blind, Clinical TrialsTrials

Double-Blind Double-Blind RandomizedRandomizedControlled TrialControlled Trial

TherapyTherapy

Search FiltersSearch FiltersMethodologyMethodologyType of Type of QuestionQuestion

EBM ResourcesEBM Resources Pocket guides with web-linked Pocket guides with web-linked

updates (Sackett; Guyatt & Rennie)updates (Sackett; Guyatt & Rennie) Cochrane LibraryCochrane Library BMJ: BMJ: www.clinicalevidence.comwww.clinicalevidence.com Centre for EBM: Centre for EBM: http://http://

minerva.minervation.comminerva.minervation.com Centre for Evidence-based mental Centre for Evidence-based mental

health: health: www.cebmh.comwww.cebmh.com

Evidence PyramidEvidence Pyramid

Evidence Pyramid - OVIDEvidence Pyramid - OVID

Evidence Pyramid - SUMsearchEvidence Pyramid - SUMsearch