measuring nonprofit outcomes
TRANSCRIPT
What Brought You Here?
Why You Should Measure Outcomes
What You Should Measure
How To Make It Happen
2
Why are you interested in this workshop?
What do you hope to take away today?
3
Program evaluation is carefully collecting and analyzing information about a program or some aspect of a program in order to make necessary decisions.
The type of evaluation you undertake to improve your program depends on what you want to learn.
Continual improvement is an unending journey. 4
Process Descriptions of what you do, how much you produce, and how you do it
OutcomesChanges that occur within the client, member or system as a result of your activities
5
6
External PressuresIncreased Public ScrutinyFunder AccountabilityEconomic Decline
Internal MotivationsContinuous ImprovementGreater Mission Focus
7
Direct service providers need to demonstrate results for the client or target populationAssociations must justify fees and retain membersSystems change nonprofits need to show changes in system capacity, investment or readinessFoundations must demonstrate impact across grants awarded or at the community level
8
9
Data can provide powerful evidence
Data can be objective and reduce bias
Balanced information provides the opportunity for independent interpretation
Data in existing tracking systems may be useful
10
Who are the possible audiences for your data?
What do they already know?
What do they want to know?
How will they use the data?
11
Possible Audience What Do They Already Know?
What Do They Need to Know?
How Will They Use the Data?
12
Exercise
Designing a good tool requires more time and attention than you may think.
Questions can tap into:• Characteristics• Opinions• Experiences• Knowledge• Attitudes• Behaviors
13
Process –activity/output/efficiencyNumber and type of volunteer hours provided, volunteers trained, workshops held, applications processed, etc.Example: Average contact time with participants
Outcomes – effectiveness/results/impactCan include change in clients’ level of awareness, knowledge gained, behaviors that have changed, etc. Example: Percentage of participants who demonstrate increased knowledge of effective parenting skills
14
What is a logic model? A logic model is a simple description of how your program, service, or initiative works that shows the linkages between:
Problem you are attempting to addressProgram componentsProgram activitiesOutcomes (both short- and long-term)
15
Useful for designing/developing your program or initiativeProvides a framework for evaluationProvides a focal point for stakeholders, requiring them to work together to identify the components of the model and to think about expected outcomes
16
17
Find a championUsually someone in leadershipIntegrate communication about the effort into operations
Select someone who can implement and propel the effort forward
Choose a person who has a compatible mindsetChoose a clear communicatorProvide professional development if neededGive authority that allows dedicated time
18
What outcomes will you measure?How will you define each outcome? What tools will you use to do so?Who is responsible for data collection?What are the timelines for doing so?What questions do you want to be able to answer at the end of the year?
19
SurveysPersonal interviewsFocus groupsOther data collections tools Existing databases
20
A quantitative research project in which a relatively large number of people are queried, each being asked a standard set of questions, posed in the same way each time.
21
To increase understandingTo collect specific, standardized data across respondentsTo collect data across many people as efficiently as possible To collect data quickly, even across multiple groupsOpen-ended quotes may add impact and credibility
22
An interview is the collection of data by asking people questions and following up or probing their answers.
23
To increase understandingAs an exploratory first step to creating quantitative toolsEnhance understanding of interesting findings which emerged from other processesReal-world quotes may add impact and credibilityTo collect specific, standardized data across respondents
24
A focus group is a group discussion. Participants are brought together in a neutral location for the specific purpose of discussing and issue or responding to ideas or materials of interest.
25
To increase understandingTo brainstorm or explore an idea you may wish to implementEnhance understanding of interesting findings which emerged from other processesTo generate ideas for possible solutions
26
ObservationsChecklists (process documentation)Attendance logsCase recordsExisting databases
27
Depends on the purpose – who can answer your questions?If not apparent, ask for guidance from key informants.If potential population is large, generate a list of candidates and key characteristics then randomly select.
28
Document your processes firstChoose the most simple solution that will get the job doneInclude leadership, data entry personnel and data analysts in these conversationsFocus on how data relates to reportingSoftware is not the silver bullet
29
Keep it simpleTake baby stepsBalance simplicity with precisionFocus on interpretationBlend methods if appropriate
30