research strand: program evaluation · 2013. 6. 24. · research strand: program evaluation stephen...
Post on 07-Feb-2021
1 Views
Preview:
TRANSCRIPT
-
Research Strand: Program Evaluation
Stephen Kroeger, Christy Borders, Kim Webster
-
LogicizingUniversity of Cincinnati
The Birth of a Logic Model to Evaluate one 325T Program
-
Answers Needed The Government Performance and Results Act (GPRA)
essentially asks three questions: What is our program trying to achieve? How will our program’s effectiveness be determined? How is our program actually doing?
The Challenge Clear and logically consistent methods have not been readily
available to help program coordinators make implicit understandings explicit
(McLaughlin & Jordan, 1999).
-
Measuring Change Providers are often being required to document specific
interventions and expected outcomes and to demonstrate their efforts to increase program effectiveness.
As a result, a great number of providers need to identify services or policies that are designed to lead to positive changes in the services provided to children and families they serve.
Yampolskaya, Nesman, Hernandez, & Koch, 2004
-
What is a Logic Model process? The Logic Model process is a tool used by program
evaluators and coordinators to develop and tell the performance story for their program.
The Logic Model describes the reasonable linkages among program resources, activities, outputs, stakeholders reached, and short-intermediate-longer term outcomes.
Once this model of expected performance is produced, critical measurement areas can be identified.
(McLaughlin & Jordan, 1999)
-
Benefits of a Logic Model process Builds a common understanding of the program, identifies
assumptions, and supports team communication Identifies projects that are critical to goal attainment,
redundant, or have inconsistent or implausible linkages among program elements
Communicates the place of a program in the organization or problem hierarchy
Provides a balanced set of key performance measurements and evaluation issues to improve data collection and usefulness
Meets requirements of GPRA(McLaughlin & Jordan, 1999).
-
Logic Model Models Gugiu & Rodriguez-Campos, 2007 Theory Approach logic model asks four questions: What issues or
problems does the program seek to address? What are the specific needs of the target audience? What are the short- and long-term goals of the program? What barriers or supports may impact the success of the program? W. K. Kellogg foundation logic models
United Way of America logic model consists of resources needed to achieve goals; activities implemented to affect change; outputs are services provided, materials distributed, people served; and outcomes are derived benefits as a result of program activities. Contextual factors may be incorporated when it’s important to note constraints on the program.
Antecedent Target Measurement (ATM) logic models identify antecedent conditions on a problem’s underlying rationale (why); identify antecedent conditions targeted by program activities; determine whether outcomes are reasonable to include in the logic model in light of the evaluation timeframe.
Open Systems logic models focus on establishing collaborative partnerships to achieve strategic objectives and to measure impacts at various levels. Levels include changes that are temporary; sustaining the prior changes and stakeholder buy-in; making broader changes in organizational practices prevent/reduce problems.
-
Learning Curve During Construction Activities are not Goals We had over 10 activities we thought were goals These activities were categorized into 4 main goals
Outcomes Link Back to Activities Whether Short term, Intermediate or Long term, the
outcomes need to provide data used to evaluate the activity Outcomes Must be Measureable If we can’t quantify or qualify the outcome we won’t know if
the activity was useful or effective The act of establishing outcome measures required that we
modify the activities to produce measureable outcomes What activity would increase retention of new teachers?
-
Logic Model Changed Our Thinking
-
Logic Model Changed Our Thinking
-
Logic Model Changed Our Thinking
What productwill come outof this activity?
Changes orbenefits causedby outputs
-
Logic Model Changed Our Thinking
Short-term = changes or benefits causedby outputs.
Intermediate = results as an applicationof short-term
Long-term = are these results correct?
-
Logic Model Changed Our ThinkingData: SurveyCollection: YearlyFeedback: Failure to meet outcome would result in the examination of how we are communicating instruction on EBP
What data?
When/how to collect?
How will this feedback to activities?
-
Daily Use of the Logic Model
How can we use the Logic Model to inform day-to-day decisions?
Funneling the goals into daily tasks in a manageable format
GoalsActivitiesDaily Tasks
-
Feedback Loops
Data provides continuous feedback. Resources,activities, outputs, customers, or outcomes may adjust based on the how the data are supporting the project’s goals.
-
Sample of Daily Graphic
-
Questions How does a logic model help me? What resources do I need in order to begin using a
logic model? Can you give an example of how the LM helped you
in your project design and evaluation?
-
References Gugiu, P. C., & Rodriguez-Campos, L. (2007). Semi-
structured interview protocol for constructing logic models. Evaluation and Program Planning, 30, 339-350.
McLaughlin, J. A., & Jordan, G. B. (1999). Logic models: A tool for telling your program's performance story. Evaluation and Program Planning, 22, 65-72.
Yampolskaya, S., Nesman, T. M., Hernandez, M., & Koch, D. (2004). Using concept mapping to develop a logic model and articulate a program theory: A case example. American Journal of Evaluation, 25(2), 191-207.
Research Strand: Program EvaluationLogicizing�University of CincinnatiAnswers NeededMeasuring ChangeWhat is a Logic Model process?Benefits of a Logic Model processLogic Model Models �Gugiu & Rodriguez-Campos, 2007Learning Curve During ConstructionLogic Model Changed Our ThinkingLogic Model Changed Our ThinkingLogic Model Changed Our ThinkingLogic Model Changed Our ThinkingLogic Model Changed Our ThinkingDaily Use of the Logic ModelFeedback LoopsSample of Daily GraphicQuestionsReferences
top related