partnership design and evaluation...partnership design and evaluation mnscu webinar june 17, 2015...
TRANSCRIPT
Partnership Design and Evaluation
MNSCU Webinar June 17, 2015
Presented by
Brett Pawlowski
Every Teen with a Dream and a Plan
Every Community with a
Capable, Ready Workforce
MISSION
Resources from NC3T
www.Amazon.com
www.ACTEOnline.org
www.ACTEOnline.com
• Partnership essentials
• Partnership models
• How to recruit
• Measurement
• Sustainability
• Advisory boards
• Case studies
Today’s Agenda
• Goal Setting
• Partnership Design
• Evaluation
I. Goal Setting
Why Set/Measure Goals?
• Make sure partnership is
accomplishing something real:
Not just “feel good”
• Prove the value to stakeholders
• Benchmark/Improve
• Encourage future partnerships
Why Set/Measure Goals?
• Partnership Selection Criteria–Willingness to collaborate: 84.7%
– Interest in measurable outcomes: 67.6%
– Commitment to the project: 64.0%
–Needs of the student body: 48.6%
– Existing relationship with coalition: 33.3%
–Accessibility to location: 21.6%
Source: Coalition Leaders Speak Out on Education, DeHavilland Associates, 2007
Case Study: DARE
• Common sense didn’t lead to
real-world outcomes
•Measurement provided a
chance to
fix things
SMART Goals
Goals for Each Stakeholder
• Student outcomes: first and
foremost
• Also … business partner goals
• Also … school/program goals
Sample Student Goals
• Increase graduation rate to 80%
• Increase number of students
receiving industry certifications
to 40%
• Increase student understanding
of an industry/occupation
More on Student Goals
• Research and
case studies at
NC3T.com
Sample Business Goals
• Hire five qualified machinists from the program each year
• Improve soft skills of the local labor pool
• Boost employee morale
Sample Education Goals
• Improve measured education statistics (grad rates, state assessments, etc.)
• Keep teachers/professors current on industry expectations
• Improve student engagement (attendance, participation, etc.)
Interim Goals
• Improving postsecondary completion rates = long term
• Points along the path?–Student/parent awareness
–FAFSA application
–Earning postsecondary credits
–Industry connections
Final Thought
There is a difference between
what is easy to
measure and what
is important
to measure
II. Partnership Design
The Logic Model
Lays out the thinking that drives your partnership
1. Where you’re starting
2. What is your
intervention
3. What is your target
The Logic Model
Development Order:
3, 1, 2
1. Where you’re starting
2. What is your
intervention
3. What is your target
End Point, Start Point
• End point: already set through goal
setting process (for all stakeholders)
• Starting Point:
Where are you
starting from?
Intervention
• How are you going to get from here to
there?
Intervention
Three components:
• Resources
• Activities
• Participants
Resources
What it takes to fuel your partnership:
• Administration
• Facilities/transportation
• Funding
• Expertise
Activities
Nine partnership models:
• Advisory boards
• Expertise
• Mentors
• Real-world challenges
Activities
• Student work-based learning
• Teacher learning experiences
• Advocacy
• Resource support
• Program start-up or retrofit
Participants
Stakeholders and Roles
• Students
• Volunteers/mentors
• Administrators
• Parents
The Full Logic Model
Logic Model: An Example
You and your partner(s) decide:
• Grad rate to 85%
• 70% of grads pursue or
complete postsecondary
education
• 35% go into manufacturing
• Partners hire 15 students next
year, 20 the year after, and 25
the year after that
Logic Model: An Example
Where are you starting?
• Grad rate at 80%
• 60% of grads pursue
postsecondary education
• 25% go into manufacturing
• Partners hire 10 students/year
Logic Model: An Example
You look at your resources:
• Administrative support
• Facilities/transportation
• Business partners who can
offer volunteers, mentors, and
site visits
• Partner funds for incidentals
Logic Model: An Example
You decide on activities:
• Career exploration for all
students – guest speakers, site
tours, career fairs, online
exploration
• Career mentoring for select
students in upper grades
Logic Model: An Example
You identify participants:
• All students grades 10/11
• Select students grade 12
• 30 company employees (10 for
exploration, 20 for mentoring)
• Administrators (business/ed)
• Parents
Logic Model: An Example
III. Measurement/Evaluation
ROI: Return on Investment
What do your stakeholders
put in?
What do they
get out?
Two Types of Data
Qualitative = “Words”
• Focus groups
• Interviews
• Observations
• Journals
Two Types of Data
Quantitative = “Numbers”
• Number of students hired
• Graduation rate
• Attendance figures
• Test scores
• Survey results
Student Outcomes
• School-based statistics
– Graduation rates
– Attendance rates
– Incident rates
– Scores on state assessments
– Other official assessments (WorkKeys, SAT,
ACT, etc.)
– Student grades
Student Outcomes
• Employability skills
• Industry certifications
• Work experience
• Work product (projects, portfolios)
• Extracurriculars
• Course selection
• Education/career plans
Business Outcomes
• Hiring
• Labor pool measurements – quality of
graduates
• Employee numbers
– Morale
– Retention
– Skill development
• Branding/Awareness
Education Outcomes
• Student outcomes
• Teacher measures
– Quality (certifications?)
– Morale
– Retention
• Resources/Volunteer hours
• Program performance
What Do You Do with This Data?
• Share it!
• Keep getting
better!
Effective Practices
• Don’t overreach
• Get a baseline
• Measure consistently
• Train your people
• Review and modify
your program based
on data
Effective Practices
For larger-scale initiatives:
• Independent evaluation
• Random assignment
• Test/control model
Questions?
Type questions directly into panel box
