optimizing your debis: ways to enhance the success of evidence-based interventions
TRANSCRIPT
![Page 1: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/1.jpg)
Optimizing Your DEBIs: Ways to Enhance the Success of
Evidence-Based Interventions
Marlene Glassman, Ph.D.Prevention Research Branch
Centers for Disease Control and Prevention
National HIV Prevention ConferenceAugust 15, 2011
![Page 2: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/2.jpg)
The findings and conclusions in this presentation are those of
the author and do not necessarily represent the views
of the CDC
![Page 3: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/3.jpg)
“If you don't know where you are going, any road will get you there.”
Lewis CarrollAlice's Adventures in Wonderland
![Page 4: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/4.jpg)
Optimization Requires an Inquiring Mind
• Where Do You Want To Go With Your DEBI?
• What Do You Want To Accomplish?• What Else Do You Want To Know?• How Will You Know It?
![Page 5: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/5.jpg)
Why All the Questions?
Accountability• Board of Directors• Funders (CDC, Congress)• National HIV/AIDS Strategy• Local elected officials• Fiscal responsibility/taxpayers’
money/competition for scarce resources• CLIENTS
![Page 6: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/6.jpg)
Key Questions
• Resources• Client recruitment• Characteristics of clients (did we reach target
population?)• Extent of client participation• How intervention was delivered
IMPLEMENTATION• Client outcomes
![Page 7: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/7.jpg)
IMPLEMENTATION
“Over the past decade, the science related to developing and identifying ‘evidence-based practices and programs’ has improved –however, the science related to implementing these programs with fidelity and good outcomes for consumers lags far behind.”
Dean L. FixsenThe National Implementation Research Network2005
![Page 8: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/8.jpg)
How Can We Assess Implementation?
• Operational research• Implementation research• Process monitoring• Process evaluation
![Page 9: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/9.jpg)
Operational Research
The study of factors that affect the implementation of HIV prevention programs and the subsequent
identification and assessment of recommendations to strengthen programs in
order to reduce HIV/AIDS incidence, morbidity, and mortality
![Page 10: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/10.jpg)
Operational Research Questions
Overriding Questions• What factors affect the delivery of DEBIs by
community-based organizations (CBOs)?• What factors are associated with intervention
success?• How can implementation be improved so that
more clients achieve the intended outcomes of the intervention?
![Page 11: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/11.jpg)
Operational Research Questions
More Specific Questions
• Do CBOs select appropriate DEBIs to meet the needs of their target populations?
• Do CBOs adapt DEBIs in accordance with CDC guidelines?
![Page 12: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/12.jpg)
Implementation Research Questions
• Have DEBI facilitators received required training on the intervention?
• Do CBOs implement DEBIs in accordance with the protocol and core elements of the intervention?
• How do CBOs assess whether DEBIs are implemented according to the protocol and core elements of the intervention (quality assurance)?
• Do CBOs have/implement client recruitment and retention plans?
• Do CBOs have/implement monitoring and evaluation plans?
![Page 13: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/13.jpg)
Process Monitoring Questions
• How many clients did we recruit?• What are their characteristics (e.g., age,
race, gender) and risk factors?• How many clients completed all sessions?
Less than half of the sessions? Only one session?
![Page 14: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/14.jpg)
Process Evaluation Questions
• Did recruitment take place according to client recruitment and retention plan/SMART objectives?
• Did retention take place according to client recruitment and retention plan/SMART objectives?
• Are expenditures in line with projections and budget?
![Page 15: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/15.jpg)
Outcome Monitoring QuestionsOutcomes depend on implementation: that’s
what all the fuss is about!*• Did clients achieve the immediate and behavior
change outcomes identified in the DEBI’s monitoring and evaluation plan (e.g., immediate: increased knowledge and self-efficacy; behavior change: increase in correct condom use)?
• Were outcomes measured with appropriate instruments?
*assumption that efficacy = effectiveness
![Page 16: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/16.jpg)
What You Need to Answer Questions
• DEBI budget• Behavior change logic model• Protocol/curriculum with core elements• Client recruitment and retention plan• Monitoring and evaluation plan with SMART
objectives and instruments to measure change• Quality assurance plan (fidelity checklist to
assess fidelity to core elements)
![Page 17: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/17.jpg)
An Inquiring Mind
• Whether you perform operational research, implementation research, process monitoring, or process evaluation, maintain an inquiring mind and carry out activities to answer key implementation questions.
• Remember: without proper implementation you reduce the potential for clients to achieve optimal outcomes.
![Page 18: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/18.jpg)
Summary Checklist
Trained facilitators and other staff; e.g., outreach workers (client eligibility), supervisors (quality assurance)
Implementation according to intervention protocol and core elementsSMART process and outcome objectives– Recruitment of target population– Retention of target population– Changes in clients
Financial oversight
![Page 19: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/19.jpg)
Knowing Where You’ve Been Can Help You Change Direction
(when needed)
![Page 20: Optimizing Your DEBIs: Ways to Enhance the Success of Evidence-Based Interventions](https://reader034.vdocuments.us/reader034/viewer/2022052622/558d06e2d8b42a54668b46f4/html5/thumbnails/20.jpg)
“I haven't failed. I've just found 10,000 ways that won't work.”
Thomas Edison