©performance results, inc. developing and using logic models for project planning and evaluation...
TRANSCRIPT
![Page 1: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/1.jpg)
©Performance Results, Inc.
Developing and Using Logic Models
for Project Planning and Evaluation
NIH Science Education Projects Annual Conference
May 15, 2012
![Page 2: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/2.jpg)
Why Outcomes Matter
Constructive FailureUnconstructive Failure
http://www.philanthropyroundtable.org/topic/excellence_in_philanthropy/failure_in_philanthropy
Results-based Evaluation
![Page 3: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/3.jpg)
Three buckets of data….
Evaluations
Process
PeopleResults
![Page 4: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/4.jpg)
The Evaluation Process Should Answer:
How has my program made a difference?
How are the lives of the program participants improved as a result of my program?
Results-based Evaluation
![Page 5: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/5.jpg)
![Page 6: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/6.jpg)
What are Outcomes?
A target audiences’ changed or improved skills, attitudes, knowledge, behaviors, status, or life condition brought about by experiencing a program.
Results-based Evaluation
![Page 7: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/7.jpg)
What is a Program?
Activities and services leading towards intended outcomes
Has a definite beginning and end Designed to change attitudes, behaviors, or
knowledge, or increase skills and abilities related to assumed need
Results-based Evaluation
![Page 8: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/8.jpg)
Program Purpose
Program purpose is driven by assumptions about need. It relates to the organization’s mission statement. It defines audience, services, and outcomes.
In summary...We do what, for whom, for what outcome or benefit?
Results-based Evaluation
![Page 9: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/9.jpg)
What: We provide assessment, goal development and one-to one tutoring
To whom: Adults living in Reynolds County and surrounding area who express an interest and demonstrate a need for our services
For what outcomes: Adults will improve their reading, writing and communication skills and achieve their personal literacy goals
Program Example: Literacy
Results-based Evaluation
![Page 10: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/10.jpg)
What: We will establish a community-wide partnership with health professionals, teachers, parents, children, health researchers and the media to provide raise awareness about obesity among children, promote healthy diets and lifestyles, create family health plans and support parents efforts to help their children reach their diet and fitness goals.
To whom: Parents, children Pre-K and K, and community members in Springfield
Results-based Evaluation
Fit from the Start
![Page 11: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/11.jpg)
For what outcomes: 1. Children understand the dangers of bad eating
habits and a sedentary lifestyle2. Children improve their diet3. Children adopt an active lifestyle4. Families support one another in maintaining a
healthy lifestyle5. Families create, use and adopt a regular healthy
meal and exercise plan 6. The community demonstrate a greater
understanding of the cost of poor dietary and exercise habits of its citizens
7. The community make a commitment to sustaining programs to help families become and stay healthy and active
Results-based Evaluation Fit from the Start
![Page 12: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/12.jpg)
Logic Models
The Evaluation Plan:
An outcomes logic model (evaluation plan) is a clear, graphic representation of the links between program activities, the results these activities produce, and how the results will be measured.
![Page 13: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/13.jpg)
Inputs Activities Outputs Outcomes Impact
Resources; people,
materials, supplies, money
Program Process
Products “Counts and Amounts”
Knowledge, Skills, Behaviors,
Attitude changes of targeted
population
Community-level change
EXAMPLE 1
Logic Models
![Page 14: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/14.jpg)
Impact Outcome Indicators Inputs Activities
CommunityResults
Intended individual
change
Observable and measurable behaviors or conditions
Resources; people,
materials, supplies, money
Program Process
EXAMPLE 2
Logic Models
![Page 15: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/15.jpg)
Outcome Indicators Applied to Data Source Data Interval Target
Intended individual
change
Observable and
measurable behaviors or conditions
The population(s)
being measured
Information sources
When information is
collected
The #/% who will achieve
the outcome
The Practical Model
Logic Models
Inputs Services OutputsResources;
people, materials, supplies, money
Program Process
Products “Counts and Amounts”
![Page 16: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/16.jpg)
Start the Plan with Outcomes First
A target audiences’ changed or improved skills, attitudes, knowledge, behaviors, status, or life condition brought about by experiencing a program.
Outcomes
Outcomes Indicators Data Source Applied to Data Interval Target
![Page 17: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/17.jpg)
Outcomes are Logical and Sequential
Immediate Intermediate Long-term IMPACT
Outcomes
Outcomes Indicators Data Source Applied to Data Interval Target
![Page 18: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/18.jpg)
Outcomes
Outcomes Indicators Data Source Applied to Data Interval Target
Fit from the Start1. Children understand the dangers of bad eating habits
and a sedentary lifestyle2. Children improve their diet3. Children adopt an active lifestyle4. Families support one another in maintaining a healthy
lifestyle5. Families create, use and adopt a regular healthy meal
and exercise plan 6. The community demonstrate a greater understanding
of the cost of poor dietary and exercise habits of its citizens
7. The community makes a commitment to sustaining programs to help families become and stay healthy and active
![Page 19: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/19.jpg)
Indicators Measure the Outcomes
State the measurable conditions or behaviors chosen to show an outcome was achieved.
Show what you hope to see or know Are observable evidence of
accomplishments, changes, or gains.
Indicators
Outcomes Indicators Data Source Applied to Data Interval Target
![Page 20: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/20.jpg)
Outcome # 1 Indicator:Children understand the dangers of bad eating habits and a sedentary lifestyle
The #/% of children who can identify at least 2 poor eating habits that harm one’s health and
The #/% of children who can identify at least 3 consequences of an inactive lifestyle
Indicators
Outcome # 2 Indicator:
Children improve their diet
The #/% of children who eat 80% of the six food groups at least three times a week
Outcome # 3 Indicator:
Children adopt an active lifestyle
The #/% of children who exercise or engage in active play for 30 minutes at least 6 times a week
Fit from the Start
![Page 21: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/21.jpg)
Outcome # 4 Indicator:Families support one another in maintaining a healthy lifestyle
The #/% of families who have more than one family member supporting the child in the Fit from the Start program
Indicators
Outcome # 5 Indicator:
Families create, use and adopt a regular healthy meal and exercise plan
The #/% of families who take an active role in meal planning and choosing foods which are healthy
The #/% of families who plan and have outings and activities at least 4 times a week month
Fit from the Start
![Page 22: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/22.jpg)
Outcome # 6 Indicator:The community demonstrates an understanding of the cost of poor dietary and exercise habits of its citizens
The #/% of community members who accurately complete the online survey on fitness and diet
Indicators
Outcome # 7 Indicator:
The community makes a commitment to sustaining programs to help families become and stay healthy and active
The #/% of project partners who complete their obligations to this project and
The #/% of project partner who continue their efforts beyond the grant period
Fit from the Start
![Page 23: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/23.jpg)
Data SourcesData captured at critical points in time
Look at indicators to see what data sources are required.
Follow up – Continued Benefit
End – Results - Outcomes
Middle - Progress
Beginning - Baseline
Outcomes Indicators Data Source Applied to Data Interval Target
![Page 24: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/24.jpg)
Data Sources
Used to assess:
Behaviors
Life Condition-Life Status
Attitudes
Skills
Knowledge
Outcomes Indicators Data Source Applied to Data Interval Target
![Page 25: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/25.jpg)
Where Will the Information Come From?
Data sources are tools, documents, and locations for information that will show what happened to your target audience. Pre-post test scores Program records Assessments Records from other organizations Observations Portfolio review
Data Sources
Outcomes Indicators Data Source Applied to Data Interval Target
![Page 26: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/26.jpg)
What kind of data do you need?
Often data sources are categorized into types:
Primary – Information you have collected yourself Secondary – Existing information collected from
someone else Qualitative – Types of information that is describing
quality rather than quantity – answers questions concerning “how”
Quantitative – Information concerning counts, amounts, percentages; answers questions concerning “how much”
Data Sources
Outcomes Indicators Data Source Applied to Data Interval Target
![Page 27: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/27.jpg)
Applied to: Target Audience
Who Gets Measured?
The target audience to whom the indicator applies
Decide if you will measure all participants, completers of the program, or another subgroup
Special characteristics of the target audience can further clarify the group to be measured
Outcomes Indicators Data Source Applied to Data Interval Target
![Page 28: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/28.jpg)
Data Intervals
When to Measure?
Data intervals are the points in time when data are collected:
Data can also be collected at the end of an activity or phase and at follow-up
Data are usually collected at program start and end for comparison when increases in skill, behavior, or knowledge are expected
Outcomes Indicators Data Source Applied to Data Interval Target
![Page 29: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/29.jpg)
Targets
Targets of Performance
Targets, also known as goals, are the stated expectations for the performance of outcomes.
Stated in terms of a number and/or percent Meets influencers’ expectations May be estimated by the program’s past
performance
Outcomes Indicators Data Source Applied to Data Interval Target
![Page 30: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/30.jpg)
Logic Models
Inputs:
The resources you will dedicate to your program.
Staff Facilities Equipment Consultants Materials Curricula
![Page 31: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/31.jpg)
Logic Models
Inputs:
Staff – researchers, health professionals, sports and dance teachers, public relations and web development team
Hands-on activities and curricula Healthy Lifestyle curricula Informal science education PSA’s Print media Website
Fit from the Start
![Page 32: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/32.jpg)
Logic Models
Services:
Services are activities, tasks and events that engage the target audience. Services are designed to have a direct effect on program outcomes.
Workshops Programs Mentoring Classes Training Job Try-outs
![Page 33: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/33.jpg)
Logic Models
Services:
Case assessment Diet and exercise plan and monitoring Age appropriate games and activities Field trips Hands-on experiences Dance and sports orientation Website development Campaign to promote healthy diets and
lifestyles
Fit from the Start
![Page 34: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/34.jpg)
Logic Models
Outputs:
A direct program product, typically measured in numbers:
# Participants # Education sessions
# Plans developed # Online surveys taken
# Plans completed # Activities completed
# Field trips # Non-completers
![Page 35: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/35.jpg)
Characteristics of People Served
Age Race Health Status Income Urban vs. Rural Primary Language Transportation Education Level
Logic Models
![Page 36: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/36.jpg)
Reporting
In summary:
We wanted to do what? We did what? So what?
![Page 37: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/37.jpg)
Three buckets of data….
Evaluations
Process
PeopleResults
![Page 38: ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15,](https://reader036.vdocuments.us/reader036/viewer/2022062511/55162d33550346b2068b4b2b/html5/thumbnails/38.jpg)
Success….
Claudia B. HornPerformance Results, Inc.
20203 Goshen Road Suite 314
Gaithersburg, Maryland 20879
www.performance-results.net240-207-3071
www.resultsonline2.net