A Structural HIV Prevention Intervention to Help CBOs

Implement Evidence-Based Interventions Effectively

Susan M. Kegeles, PhD,

Gregory Rebchook, PhD,

& the TRIP team

CAPS/University of California, San Francisco

Funded by NIMH

Translating Research Into Practice

• Substantial resources have been spent on developing and testing HIV prevention interventions: has resulted in a number of evidence-based interventions

• But until interventions are effectively implemented, the research has little impact on the epidemic

• This project focuses on how to get an evidence-based intervention put into practice in the “real world”

The Mpowerment Project (MP)• Demonstrated effectiveness

– Listed in CDC Compendium of HIV prevention interventions with evidence of effectiveness

– Scientifically tested in several communities

• Community-level intervention with multiple components

• Many CBOs want to implement the MP & call for assistance, training

Time for a New Type of Intervention

• Previous approach to influencing epidemic: develop an intervention to help change young MSM’s sexual risk behavior (MP)

• Structural intervention is needed: to impact the way that HIV prevention is conducted by CBOs

• Now: develop an intervention aimed at helping CBOs to implement MP effectively & with fidelity to MP’s core elements and guiding principles

Developing Intervention• Began developing ideas about components by

observing CBOs when we were testing the MP

• Observed CBOs that took over MP as we would leave the field in researching it

• Provided help to CBOs implementing MP

• Developed preliminary replication package (CDC’s REP project - “Replicating Effective Programs”) & tested it with CBO

Lessons Learned for Intervention - partial list

- Staff turnover is rapid• Often there is no organizational memory of prevention

program after staff leave

– Some staff are technologically savvy; others less so – CBOs can’t always afford up-to-date technology– Need to build rapport, over time, so that trust

develops– CBOs don’t want to be told exactly what to do and

how to do it • Need to give suggestions, framework• Need to be able to modify the intervention for community

and cultural context

Lessons Learned (cont.)• CBOs want to hear what worked and didn’t work

when we conducted research on MP

• Staff often don’t reach out for technical assistance (TA) early, but rather, often wait until crisis hits– Staff may not recognize when need help– Need to reach out to them, so that when crises do hit, can

provide help – Or better yet - avoid crises

• Diverse ways of learning – visual information, didactic presentations, experiential

learning - multiple methods aid in learning

• This study: to evaluate how well the Mpowerment Project Technology Exchange System (MPTES) works in helping organizations implement the MP

• This presentation: to evaluate how the MPTES is utilized by CBOs and their attitudes towards its different components - preliminary results

Methods– Provide MPTES to CBOs and examine

uptake of the various components– N = 52 CBOs’ data presented here– Will follow 70 CBOs over 2 - 3

follow-ups, at 6 month intervals– CBOs are interviewed by phone at

each assessment– This presentation: examine utilization

patterns after CBOs have MPTES for at least 6 months (follow-up 1)

– Study evaluator interviews EDs, project supervisors, & coordinators

The Mpowerment Project Technology Exchange System

(MPTES)• 7 components:

– Program Manual – Program Overview Video– Small group facilitator’s manual– Small group training video– 3-day training– Technical Assistance (TA)– Internet Resources (website focused on here)

Program Manual• Developed after input by

former staff, CBOs using it, advisory board comprised of CBOs from around SF/Bay Area

• Lots of visuals

• Carefully organized

• Very user friendly

Program Overview Video &

Small Group Video

• To give a “feel” for what intervention is like

• Filmed on location when conducting intervention in Albuquerque

• Overview video: to use with funders, boards of directors, agency when deciding if to implement MP, sometimes core groups (initially)

• Small group video: to train small group facilitators, to be used with facilitator’s manual for small groups

Training• Designed for project staff

and supervisors• Experiential and

interactive• Conducted by original

researchers and former Project Coordinators

• 3-days long• In-depth• Comprehensive • Information exchange

among CBOs at training

Technical Assistance (TA)• Collaborative Approach

• TA to coordinators & supervisors

• Pro-active– TA providers call each CBO every 2 to

3 weeks– CBOs can request TA at other times

• Tailored – TA helps CBOs adapt MP to the needs

of their community

• Credible Providers– Former MP Coordinators– Experienced implementing MP both

during the research period and in a CBO

www.mpowerment.org• Design complements the

training manual• Online modules

– HTML– PDF

• Message board• E-mail forum• Chat room• Examples of outreach

materials (over 100)• Links and team photos

Evaluation of Program Manual• Range of use: 0 - 41 times,

median = 9.25

• attitude scales, 1 - 6, with 1= strongly disagree and 6= strongly agree (medians)

• Helpful to set up: 5.46• Helpful reference: 6.00• Still use it over time: 6.00• Helpful at tailoring: 5.00

• their “purple bible”...they used it to help them recruit staff, to keep on track with the community assessment, to keep the goals and objectives in the forefront of their thinking...it has been indispensable for them

• “the keystone training prevention manual that all prevention manuals should be measured against in the future... and I look at a lot of manuals…”

• he knows that Joey doesn’t like it because it is overwhelming...Jason tries to get him to look at it more...but he doesn’t

Evaluation of Overview Video

• Range of use: 0 - 15 times, median = 1.00

• Helpful to inform Core Group: 5.33

• Helpful to understand “look” of program: 5.66

• Teach coordinators: 5.55• Still use: 4.04

• good to show in the community ...short and sweet...gives a good overview and good to show potential members of a core group, providers

• loves the video... thinks it is so cute...it’s great...he always tears up when they show the graphs about what impact it had...he is craving that so much in his work...

• he didn’t think that some of the stuff was right for their culture...the African American guy in the film wasn’t like they are...he thinks we could make it better by seeking more information from the inner city...

Evaluation of Training• Range: 0 - 1 times, median

= 0 (but most went to training before baseline)

• Helpful to talk with others doing MP 6.00

• Helped to decide to implement: 5.50

• Enjoyed 5.75• Adjusted approach

because of training 5.75

• training was excellent...it kept his attention for the entire three days

• it was great to touch base with all the people who were implementing and make those connections... have always wanted a training version 2.0 where the groups addressed issues that they encountered through implementation and adaptation in their communities

• the training didn’t address organizational issues.... It didn’t encourage you to get organizational buy in, and that’s important to have as well

Evaluation of TA by phone

• Range: 0 - 10 times, median = 2.00

• TA receive when call CAPS is helpful 6.00

• Helped in obtaining additional funding 4.00

• Like getting TA by phone 5.55

• Helpful when CAPS initiates calls 5.58

• when TA provider called, it was a huge relief on a psychological level to have someone on the phone who had implemented the project before......if someone calls occasionally from the outside, it makes him feel less isolated....

• they would like TA provider to visit the program....the guys get a little frustrated because sometimes the information that is provided doesn’t always hit the mark....it would be helpful if the TA provider came out to visit and saw the community and the program

Evaluation of Website

• Range: 0 - 20 times, median = 1.33

• Helpful in answering questions about MP 4.00

• Help generate ideas for events 5.00

• Help generate ideas for outreach 5.00

• the web site is nice...as a resource, it is one of the easiest to navigate...it takes you to information quicker, and because it is on line, you can word search it, which is faster [than manual]

• he downloads the modules from on line and prints them out so he can mark them up (vs. manual)

• doesn’t really use it...it’s hard to find time to fit it all in

• gets more out of speaking with TA provider than the web site

Summary of Utilization & Attitudes

• Strongly positive attitudes towards MPTES

• But utilization is substantially lower than would be expected by attitudes – possibly because staff is so busy with work that little time

to use

• Biggest issues/dissatisfactions:– Want TA providers to come to CBOs in person– Want more help in adapting intervention– Especially questions/concerns about MP and how to adapt

to African American MSM and rural areas

Acknowledgements

The translation of research into practice

(TRIP) team:

John Hamiga, David Sweeney,

Scott Tebbetts & Ben Zovod

also:

Emily Arnold, Michael Foster,

Dave Huebner, Wayne Steward

and 70+ CBOs trying to prevent HIV/AIDS

Funding: NIMH