count your impact: getting started with outcome based evaluation
TRANSCRIPT
![Page 1: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/1.jpg)
COUNT YOUR IMPACT:
GETTING STARTED WITH
OUTCOME-BASED
EVALUATION
Nicolle Steffen & Linda Hofschire
www.lrs.org | @lrs_co | ripl.lrs.org
![Page 2: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/2.jpg)
DATA TRIFECTA
Inputs Outputs Outcomes
![Page 3: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/3.jpg)
INPUT MEASURES
Quantifying the effort
expended on a program
Inputs
![Page 4: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/4.jpg)
funds expended
![Page 5: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/5.jpg)
number of staff
![Page 6: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/6.jpg)
staff hours
![Page 7: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/7.jpg)
OUTPUT MEASURES
Quantifying the level of
services and materials
provided
Outputs
![Page 8: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/8.jpg)
number of registered participants
![Page 9: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/9.jpg)
number of events offered
![Page 10: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/10.jpg)
total attendance at events
![Page 11: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/11.jpg)
number of minutes spent reading
![Page 12: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/12.jpg)
circulation
![Page 13: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/13.jpg)
OUTCOME MEASURES
Quantifying the effect a
service has on the
program’s stated objectives
Outcomes
![Page 14: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/14.jpg)
Children’s enjoyment
of reading increased
![Page 15: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/15.jpg)
children maintained
or improved reading skills
![Page 16: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/16.jpg)
children were more
likely to read by choice
![Page 17: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/17.jpg)
library developed long-term
partnership with local school district
![Page 18: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/18.jpg)
DATA TRIFECTA
Inputs Outputs Outcomes
![Page 19: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/19.jpg)
ACTIVITY: WHAT IS YOUR DATA TRIFECTA?
Inputs Outputs Outcomes
![Page 20: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/20.jpg)
ACTIVITY: WHAT IS YOUR DATA TRIFECTA?
Outcome-Based Evaluation Project
Project name: Senior Computer Class Series
Description: My library offers a class series
for seniors that covers basic computer
/technology skills
![Page 21: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/21.jpg)
Inputs Outputs Outcomes
Funds expended:
• Copies of
handouts
• Marketing
campaigns at
local senior
centers
Staff hours:
• Curriculum
development
• Instruction
Number of
events offered
Number of
registrants
Number of
attendees
Digital literacy
Confidence
using technology
Ongoing
partnerships with
local senior
centers
![Page 22: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/22.jpg)
Library A Library CLibrary B
![Page 23: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/23.jpg)
Library A
• 100% of participating children increased or maintained reading skills
• 85% of participating children increased or maintained reading skills
• 85% of participating children increased or maintained reading skills
Library CLibrary B
![Page 24: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/24.jpg)
Library A
• 100% of participating children increased or maintained reading skills
• 85% of participating children increased or maintained reading skills
• 85% of participating children increased or maintained reading skills
Library CLibrary B
• 25 children participated
• 100 children participated
• 50 children participated
![Page 25: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/25.jpg)
Library A
• 100% of participating children increased or maintained reading skills
• 85% of participating children increased or maintained reading skills
• 85% of participating children increased or maintained reading skills
Library CLibrary B
• 25 children participated
• 100 children participated
• 50 children participated
• $5,000 • $10,000 • $2,500
![Page 26: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/26.jpg)
Library A
• 100% of participating children increased or maintained reading skills
• 85% of participating children increased or maintained reading skills
• 85% of participating children increased or maintained reading skills
Library CLibrary B
• 25 children participated
• 100 children participated
• 50 children participated
• $5,000 • $10,000 • $2,500
• $200/child • $100/child • $50/child
![Page 27: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/27.jpg)
Inputs Outputs Outcomes
ACTIVITY RECAP: WHAT IS YOUR DATA TRIFECTA?
![Page 28: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/28.jpg)
IntermediateImmediate Long-term
OUTCOMES
![Page 29: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/29.jpg)
IntermediateImmediate Long-term
OUTCOMES
Learn how to complete online job application
![Page 30: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/30.jpg)
IntermediateImmediate Long-term
OUTCOMES
Learn how to complete online job application
Get an interview
![Page 31: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/31.jpg)
IntermediateImmediate Long-term
OUTCOMES
Learn how to complete online job application
Get an interview
Get a job
![Page 32: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/32.jpg)
IntermediateImmediate Long-term
OUTCOMES
Learn how to complete online job application
Get an interview
Get a job
![Page 33: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/33.jpg)
LINKING IMMEDIATE OUTCOMES TO LONG-TERM IMPACT
Reading for pleasure associated with:
Increased empathy
Improved relationships
Reduced depression symptoms
Reduced risk of dementia
The Reading Agency, 2015
![Page 34: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/34.jpg)
LINKING OUTPUTS TO LONG-TERM IMPACT
Reading more associated with…
Improved reading skills
Better development of vocabulary, writing,
and grammar
Chin & Krashen (2007)
![Page 35: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/35.jpg)
MEASURING OUTCOMES
![Page 36: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/36.jpg)
INDICATORSOutcome Indicator
Increased
enjoyment of
reading
![Page 37: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/37.jpg)
INDICATORSOutcome Indicator
Increased
enjoyment of
reading
Parent response
to survey
question
![Page 38: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/38.jpg)
INDICATORSOutcome Indicator
Increased
enjoyment of
reading
Parent response
to survey
question
Increased digital
literacy
![Page 39: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/39.jpg)
INDICATORSOutcome Indicator
Increased
enjoyment of
reading
Parent response
to survey
question
Increased digital
literacy
Demonstrate
ability to text,
send an email
with an
attachment, etc.
![Page 40: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/40.jpg)
After participating in the Summer Reading
Program, did your child’s reading habits change?
Please indicate if the following things increased,
stayed the same, or decreased for your child:
Increased
Stayed the
Same Decreased
Enjoyment of reading
Reading skills
Reading by his/her
choice
SURVEY
![Page 41: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/41.jpg)
While I was on a computer in the computer center
today, I . . .
SURVEY
Looked for employment
(job search, resume,
application, etc.).
Communicated with
someone (email, chat,
Facebook, etc.).
Used the computer for
entertainment (ex: videos,
music, games).
Used government
resources (licenses, taxes,
unemployment, disability,
welfare, immigration, etc.).
Did schoolwork. Found health information.
![Page 42: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/42.jpg)
Please let us know in your own words
the difference the 2014 Summer
Reading Program has made for your
child(ren) and family:
SURVEY—OPEN ENDED QUESTIONS
![Page 43: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/43.jpg)
As a first time parent it's really
helpful to have tangible ideas
about ways to get my child
engaged in early literacy skills.
There were some
recommended activities I hadn't
thought of trying!
![Page 44: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/44.jpg)
I loved being able to spend time with my
child and bond over my love for books.
My son's reading endurance has
really grown this summer, thanks to
the reading program. He was very
motivated to read for 40 minutes and
sometimes he read for even longer
stretches.
![Page 45: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/45.jpg)
Please tell us how the computer center
helps you or your community:
SURVEY—OPEN ENDED QUESTIONS
![Page 46: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/46.jpg)
Probably the most valuable
resource, dollar for dollar,
available to community. I have
found jobs…researched…
located tax information, and
have done school work over the
years. Thank you!
![Page 47: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/47.jpg)
The access to the internet
afforded by the public library is
most probably the only reason I
am not completely bereft of any
and all computer skills. Were it not
for this access, as well as the
assistance rendered via classes
offered, I would most likely be
unemployed if not unemployable.
![Page 48: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/48.jpg)
PLA-PROJECT OUTCOME
![Page 49: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/49.jpg)
PLA-PROJECT OUTCOME
Early Childhood Literacy
Education/Lifelong Learning
Digital Inclusion
Job Skills
Economic Development
Civic/Community Engagement
Summer Reading
![Page 50: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/50.jpg)
IMPACT SURVEY
![Page 51: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/51.jpg)
CONTINUUM
![Page 52: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/52.jpg)
CONTINUUM
![Page 53: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/53.jpg)
CONTINUUM
![Page 54: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/54.jpg)
POSTCARD
![Page 55: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/55.jpg)
PRIZE DRAWING
![Page 56: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/56.jpg)
SKILL DEMONSTRATION
![Page 57: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/57.jpg)
COMMENT WALL
![Page 58: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/58.jpg)
ACTIVITY: OUTCOME INDICATORS AND METHODS FOR MEASUREMENT
Outcome Indicators:
Outcome=Digital literacy
Indicator #1=Class attendee is able to send an
email with an attachment
Indicator #2=Class attendee is able to like the
library Facebook page
![Page 59: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/59.jpg)
ACTIVITY: OUTCOME INDICATORS AND METHODS FOR MEASUREMENT
How will you measure outcomes?
Skill demonstration (indicators listed above)
Pre-post continuum to assess confidence
using technology
![Page 60: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/60.jpg)
I’ve collected some outcome data, now how do I use it?
![Page 61: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/61.jpg)
Strategic Planning
Management
Communication
DATA USE
![Page 62: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/62.jpg)
Area of strategic focus:
Nurture children’s learning by providing
programs and services that instill a love
of reading and advance reading skills.
STRATEGIC PLANNING
![Page 63: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/63.jpg)
After participating in the Summer Reading
Program, did your child’s reading habits change?
Please indicate if the following things increased,
stayed the same, or decreased for your child:
Increased
Stayed the
Same Decreased
Enjoyment of reading
Reading skills
Reading by his/her
choice
SURVEY
![Page 64: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/64.jpg)
Implications of OBE results for:
Budgeting
Staffing
Programs (focus, number offered,
size, timing, etc.)
MANAGEMENT
![Page 65: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/65.jpg)
Library A
• 100% of participating children increased or maintained reading skills
• 85% of participating children increased or maintained reading skills
• 85% of participating children increased or maintained reading skills
Library CLibrary B
• 25 children participated
• 100 children participated
• 50 children participated
• $5,000 • $10,000 • $2,500
• $200/child • $100/child • $50/child
![Page 66: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/66.jpg)
Internal stakeholders
External stakeholders
COMMUNICATION
![Page 67: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/67.jpg)
![Page 68: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/68.jpg)
ACTIVITY: OUTCOME INDICATORS AND METHODS FOR MEASUREMENT
Identify uses for data in…
1) Strategic planning:
2) Management:
3) Communication:
![Page 69: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/69.jpg)
ACTIVITY: OUTCOME INDICATORS AND METHODS FOR MEASUREMENT
Identify uses for data in…
1) Strategic planning: One of my library’s
strategic priorities is to provide lifelong
learning. An OBE of our senior tech classes
will help us to determine whether we’re
achieving this priority or if we need to make
adjustments.
![Page 70: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/70.jpg)
ACTIVITY: OUTCOME INDICATORS AND METHODS FOR MEASUREMENT
Identify uses for data in…
2) Managing: Are we achieving our desired
outcomes? If not, do we need to adjust staff
hours, budget, etc.?
![Page 71: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/71.jpg)
ACTIVITY: OUTCOME INDICATORS AND METHODS FOR MEASUREMENT
Identify uses for data in…
3) Communicating: Our mayor recently
announced an initiative to improve the lives of
seniors in our community. This is a multi-
faceted initiative that will address health,
wellbeing, financial and digital literacy, etc. We
can demonstrate the library’s unique role in
supporting this initiative by sharing our OBE
data with her office.
![Page 72: Count Your Impact: Getting Started with Outcome Based Evaluation](https://reader031.vdocuments.us/reader031/viewer/2022030401/58ad7df61a28ab2a768b6083/html5/thumbnails/72.jpg)
THANK YOU!
Nicolle Steffen & Linda Hofschire
www.lrs.org | @lrs_co | ripl.lrs.org