Evaluation of PHO’s KT Project

Mel Brockman

June 17, 2014

Overview CIHR supplement grant obtained.

Time period: June 2012 to June 2013.

External evaluation arranged in late 2013.

Project Overview Two target audiences:

Community/general public. Decision-makers.

Goals: Promote awareness. Encourage action.

Integrated knowledge translation (iKT) approach

Key Activities

Purpose of Evaluation Assess whether…

awareness was generated; action was prompted; and iKT was achieved.

Improvement-focused.

Type of Evaluation

Process Project operations and implementation.

Outcome Short-term (learning-based). Intermediate (action-based).

Methodology Records review.

Key informant interviews (N = 17).

Online surveys: Summit participants (N = 20). PHO network (N = 7).

Key Findings – Positives

Della video a big hit.

Summit was a success despite not arriving at a national agenda.

Public opinion survey a valuable KT research activity.

Partners happy to see the Region getting involved.

Key Findings – Areas for Improvement

Planned activities with school community.

Some groups not effectively engaged.

Process-tracing issues.

iKT not really achieved.

Recommendations Create a comprehensive conceptual

table at the outset; preferably, prior to submitting the grant application.

KT activity

Rationale

Aim Target audience(s)

Target sub-group(s)

Goal(s) activity will contribute to

Recommendations Provide potential partners with enough

information to make an informed decision about whether they are willing and able, time- and resource-wise, to sign on to the grant.

Recommendations Involve committed partners and other

stakeholders in the conceptualization of the project; preferably, prior to submitting the grant application.

Recommendations Collaboratively create a “game plan”

and clearly articulate project team members’ roles and responsibilities at the outset.

Recommendations Establish process-tracing procedures at

the outset.

Recommendations Continue to strive toward better internal

communication and coordination.

Recommendations Be more specific in the identification of

target audiences and target sub-groups within them.

Recommendations Focus on the quality of the activities, not

the quantity.

Recommendations As a team, develop a project plan that is

proportionate to the time and resources that will be available across all team members.

Recommendations Continue to explore creative communication

and dissemination strategies to engage the general public in order to go beyond “preaching to the choir.”

Recommendations Give people specific options for actions

they can take to make a difference.

Recommendations Invest the most time and resources in

activities aimed at increasing health equity locally and provincially.

Recommendations Articulate the desired actions that will

be prompted by the KT activities.

Recommendations Identify at the outset what will be

measured for evaluation purposes and design measures accordingly.

Recommendations Strive to achieve higher response rates

with data collection activities.

Activity Level of Fit

Della video High

Computer game for youth High

School presentations and student forums High

Interactive social media tools (e.g., CVC) High

Health Equity Summit High

Mobilize! forum High

Dissemination of SHS report Medium

Conference presentations, journal publications Medium

Replication/public opinion survey Low

Follow-up questionnaire with those who received SHS report Low