What are we learning about training in evidence-based public health

among local practitioners?

THANKS!!

Ross Brownson Janet Canavese

Beth Baker Katie Stamatakis

Katie Duggan Carol Brownson

Paul Erwin Case Western PRC

Robert Fields UNC PRC

Carson Smith MI PRC

Rodrigo Reis U of WA PRC

Carolyn Leep (NACCHO) Jenine Harris

Rachel Tabak Chris Casey

Amy Eyler Beth Dodson

Kathleen Wojciehowski

What is Project LEAD?4 Aims:

1. Describe the evidence-base for local EBPH in the United States (esp. A-EBPs)

2. Test the effectiveness of local-level EBPH capacity building in 4 states (Train the Trainer)

3. Describe a range of local models in EBPH

4. Translate and disseminate findings to stakeholders

Selected findings (Aim 1)

Administrative evidence-based practices (A-EBPS)

Agency (health department)-level structures and activities that are positively associated with performance measures (e.g., achieving core public health functions, carrying out evidence-based interventions)

Brownson, Allen, Duggan, Stamatakis, & Erwin. Am J Prev Med, 2012

Results, 5 EBP domains Workforce development

QI (82%), EBDM (59%) Leadership

Staff participation (84%) hire w/ PH degree (36%) Organizational climate and culture

Life long learning (71%), culture supports EBDM (42%)

Relationships and partnerships Importance (92%), share resources (68%)

Financial processes Multi funding sources (96%), QI resources (55%)

Predictors of a-EBPs

Predictors of A-EBPs in the United States, 2012 (n=317)

LHD Characteristic  OR (95% CI) aOR (95% CI)

Population of Jurisdiction

   

<25,000 1.0 1.025,000 to 49,999 7.0 (3.3, 15.0)* 7.5 (3.3, 17.3)*

50,000 to 99,999 5.7 (2.7, 12.2)* 4.9 (2.1, 11.2)*

100,000 to 499,999

9.1 (4.3, 19.6)* 7.1 (3.0, 16.9)*

500,000 or larger 7.1 (3.0, 16.8)* 4.4 (1.6, 12.5)*

Early Results

Patterns of A-EBPs and use of EBDM vary significantly based on:

1. Population size served by the LHD (strongest predictors after adjustment)

a. Smaller HDs 3X more likely to be led by a nurse

2. Governance structure (state governed)

3. Age group in the 50s

Use of resources

26% of LHDs reported systematic reviews as 1 of top 3 most important resources

34% were not familiar with the Community Guide

A tale of two worlds…How LHDs learn about research findings?

How researchers perceive they most effectively reach practitioners?

1. Professional associations 1. Journal articles

2. Seminars/workshops 2. Face-to-face meetings

3. Email alerts 3. Media interviews

4. Journal articles 4. Press releases

Selected findings (Aim 2)

Train the Trainer

• November 2012, St. Louis Missouri• Recruitment of PRC’s

– Geographic distribution – PRCs with established relationships with

PHTCs or PBRNs– “Ready to hit the ground running”

Training Locations

• Case Western Reserve UniversityPrevention Research Center for Healthy Neighborhoods and PBRN

• University of MichiganPrevention Research Center of Michigan and Michigan Public Health Training Center

• University of North Carolina at Chapel HillCenter for Health Promotion and Disease Prevention and PHTC

• University of Washington Health Promotion Research Center and PHTC

Train the Trainer Process

• Partners from PRC came to St Louis for 3 day training and overview of Evidence Based Public Health course

• Conference calls with each site to set up logistics and provide technical assistance

• EBPH slides and materials for trainings given to each site

• PRC in conjunction with PBRN or PHTC put on training for local health departments in their area

Train the Trainer

• Review of Module Content

• Focus on “Process” for course delivery

Recruiting participants for the State Trainings

• Emails to various public health listservs– PBRN, LHD commissioners, directors of

nursing, etc.

• Website postings– PRC and PHTC

• Announcements and flyers at conferences• Help from state health department

**Each state had waiting lists

Participants of Trainings

• 57% program manager or coordinator, 22% top executive, administrator, or assistant director

• 2% doctoral degree, 24% public health masters, 37% other masters, 9% nursing degree, 28% bachelors degree or less

• Mean years in current position = 5.2 years, mean years in public health = 8.8 years

• Number of organizations represented at state trainings in person = 101 (OH = 24, MI = 21, NC = 26, WA = 20)

Course Format

Two locations used traditional in-person training and two used mixed modes for course delivery i.e. in-person and webinars for distance learning capabilities

– WA – 3 one-hour webinars with 2 days of in-person training

– MI – 2 onsite training days, 3 modules each day, 3 modules by webinar, 2 of which had multiple days

11 EBPH Skills Evaluated (pre-training course survey)

I Can…• develop a concise statement of the issue • conduct a community assessment• describe the issue in a quantitative way• discover what is known about the issue through the scientific literature• prioritize program or policy options that address the issue• understand economic evaluation to use findings from others to • assist with determining which intervention would be best suited for my community and

population of interest• understand economic evaluation to assist in the design and implementation of an

economic evaluation• develop an action plan for the program or policy• qualitatively evaluate the program or policy• quantitatively evaluate the program or policy• economically evaluate the program or policy 

Pre/Post Skill Abilities(course survey)

• Data show participants felt more capable of all 11 skills after the training

– Differences in ability scores between pre and post evaluations were all statistically significant using p=.01 for sites evaluated

An Example: Skills Ability for Case Western and NC combined

I can prioritize program or policy options that address the issue

I can qualitatively evaluate the program or policy

Num

ber

of

resp

onse

s

Num

ber

of

resp

onse

s

Pre

Post

PrePost

Views Related to EBDM (pre-training survey)

• I feel that I have the skills necessary for developing evidence-based interventions (No. 76, 59.4%)

• I feel that I can effectively communicate information on evidence-based strategies to policy matters (No. 54, 42.2%)

• I feel evidence-based interventions are packaged in a way that I can use them (No. 34, 26.6%)

• I feel evidence-based interventions are designed in a way to be self-sustaining (No. 30, 23.6%)

• I feel that I need to be an expert on many issues in order to effectively make evidence-based decisions (No. 22, 17.2%)

• My fears about job security prevent me from using EBDM (No. 6, 4.7%)

Which of the following would most encourage you to utilize EBDM

in your work? (pre-training survey)

• Placing a high priority on EBDM in your work

• Trainings on EBDM• Positive feedback for EBDM use

Which of the following would most encourage you to utilize

EBDM in your work? • Director’s Survey

• Trainings on EBDM• Placing a high priority on EBDM by leaders in my agency • Positive feedback for EBDM use

• Programmatic Survey • Placing a high priority on EBDM in your work• Trainings on EBDM • Positive feedback for EBDM use

Gap between importance and availability of skills

(pre-training survey)

• Economic Evaluation 4.1• Communicating research to policy makers 3.5• Evaluation designs 3.3• Adapting interventions 3.2• Prioritizing health issues 2.7

What participants most liked about the course (post-training course survey)

• The breadth of the topics truly encouraged use of evidence across all functions of a LHD, not just programs

• Being able to concentrate on learning and having a systematic approach to follow

• Presents a whole new way of thinking about public health practice

Lessons LearnedWebinars – advantages and disadvantages

Advantages• Quicker and easier for participants• Less expensive • Flexibility of combined format was appreciated by

participants

Disadvantages• Opportunity for both formal and informal interaction

among participants and between participants and instructors is much less

• Onsite instruction was more effective

Take away messages from Aim 2

• All surveys point to the need for more training in EBPH

• The pre/post evaluations received from the trainings indicate that participants feel more competent in EBPH skills after training

• EBPH trainings = improved skills, ability and increased potential for change in local health departments

Ideas going forward…

Continue analyses, featuring more complex data linkage Explore context and meaning of Aim 1 gaps

Develop new approaches for these populations (e.g., findings for nurses)

Explore new approaches for information seeking Enhance Aim 2 additional technical assistance (beyond

the EBPH courses) Scaling up findings from Aim 2

Use Aim 3 findings to find and test new leverage points Link more closely with AcademyHealth for dissemination