integrating programintegrating program planning and ... · you need this for “organizational...
TRANSCRIPT
![Page 1: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/1.jpg)
1
Integrating ProgramIntegrating Program Planning and Evaluation Using Logic Models
BBy:Thomas J. Chapel, MA, MBAChief Evaluation [email protected]
Objectives: Be able to…
State CDC Evaluation Framework steps and standards
State the central role of program description in evaluation and planning
Describe the basic components of a logic model
2
Describe the added value of elaborating logic models
State how logic models help guide questions in program planning and evaluation
![Page 2: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/2.jpg)
2
Key Take-Aways1. It’s never about the model, it’s about
“understanding your program”2. You need this for “organizational learning”
and not just “accountability”3. Most/best benefits may be process use
benefits4. You know lots about your program even
3
before you draw your model5. A little bit of logic modeling goes a long
way. There’s a trade-off between “accuracy” and “utility”
Integrating Processes to Achieve Continuous Quality Improvement
Continuous Quality What do What do Continuous Quality Improvement (CQI) cycle
Planning—What actions will best reach our goals and objectives.
What do What do we do?we do?
Why are Why are we we
doing doing ll ll
How do we How do we do it?do it?
4
Performance measurement— How are we doing?Evaluation—Why are we doing well or poorly?
well or well or poorly?poorly?
How are How are we we
doing?doing?
![Page 3: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/3.jpg)
3
“Research seeks to prove “Research seeks to prove, evaluation seeks to improve…”
M.Q. PattonQ
Enter the CDC “Evaluation” Framework
6 6
![Page 4: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/4.jpg)
4
Enter the CDC Evaluation Framework
Good M&E= use of findings
7
Focus is situation -specific
Enter the CDC Evaluation Framework
Good M&E = use of findings Early steps
key to best focus
8
Focus is situation -specific
![Page 5: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/5.jpg)
5
Integrating Program Planning and Evaluation
Constructing Simple LogicConstructing Simple Logic Models
You Don’t Ever Need a Logic Model, BUT, You Always Need a Program Description
Don’t jump into planning or evaluation without clarity on:
The big “need” your program is to addressThe key target group(s) who need to take actionThe kinds of actions they need to take (your intended outcomes or objectives)
10
outcomes or objectives) Activities needed to meet those outcomes “Causal” relationships between activities and outcomes
![Page 6: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/6.jpg)
6
Logic Models and Program Description
L i M d lLogic ModelsGraphic depictions of the relationship between your program’s activities and its
11
p gintended effects
“Complete” Logic Model
IntermediateShort term Long-term
Activities Inputs Outputs Intermediate Outcomes
Short-term Outcomes
gOutcomes/
Impacts
What the program does…
Who or what will changebecause of the program…
What the program
1212
p gneeds…
Context and Assumptions
External factors that influence getting to outcomes
![Page 7: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/7.jpg)
7
What the program and its staff actually do
Activities Inputs Outputs
Intermediate Effects/
Outcomes
Short-term
Effects/ Outcomes
Long-term Effects/
Outcomes
13
ContextAssumptions
Results of activities:Who/what will change?
IntermediateShort Long term
Activities Inputs Outputs
Intermediate Effects/
Outcomes
Short-term
Effects/ Outcomes
Long-term Effects/
Outcomes
14
ContextAssumptions
![Page 8: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/8.jpg)
8
Finding Activities and Outcomes
Actions/n
GoalsObjectivesActions/Tactics
Activities ST or MT Outcomes
LT Outcomes or Impacts
Pla
Eval
15
p
Process MeasuresProgress Measures
Impl. Measures
Outcome MeasuresImpact Measures
Key Performance IndicatorsSuccess Factors
PM
Goal 3: Disseminate information to guide gpolicy, practice, and
other actions to improve the nation’s
health
![Page 9: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/9.jpg)
9
Finding Activities and OutcomesOffice of Workforce and Career Development
(OWCD) Mission
To improve health outcomes by developing a competent, sustainable and diverse public health workforce
through evidence-based training, career and leadership development,
17
p p ,and strategic workforce planning.
17
Inputs Activities Outcomes
Implicit Logic Model
Conduct trainingDo career leadership
developmentCompetent, sustainable,
diverse workforce
Improved health
outcomesDo Strategic workforce
Evidence Base
18
planning
![Page 10: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/10.jpg)
10
Process Use
When influence/program improvement comes not from findings of an evaluationcomes not from findings of an evaluation, but from insights gleaned during the tasks involved in doing an evaluation
Process Use in Theory, Research, and Practice:New Directions for Evaluation, No. 116, 2008
19
Underlying Logic of CPPW
Activities and
Supports
Changes in
• Policies• Systems• Environments
Changes in
• Physical activity• Nutrition• Tobacco use• Exposure to tobacco smoke
Reductions in
• Obesity• Smoking rates• Morbidity• Mortalitytobacco smoke
• Use of quit lines
Mortality
![Page 11: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/11.jpg)
11
Insights from Simple Logic Models
“Sphere of control” versus “Sphere of i fl ”influence”
Sequence of outcomes
Mismatches of activities and outcomes
“Accountable” outcomes
Frame of reference for more detailed models21
Underlying Logic of CPPW—Key Initial Outcome
Activities and
Supports
Changes in
• Policies• Systems• Environments
Changes in
• Physical activity• Nutrition• Tobacco use• Exposure to tobacco smoke
Reductions in
• Obesity• Smoking rates• Morbidity• Mortalitytobacco smoke
• Use of quit lines
Mortality
![Page 12: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/12.jpg)
12
“Nested” Logic Models—Logic Models at Different Levels of the Organization/Effort
Enterprise Level
Division/PartnerLevel
Each logic model is built with reference to the levels above and below…
23
Division/PartnerLevel
Project Level
Integrating Program Planning and Evaluation
General Underlying Logic ofGeneral Underlying Logic of FOA
![Page 13: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/13.jpg)
13
Constructing Logic Models: Identify Activities and Outcomes by….
1. Examining program descriptions,1. Examining program descriptions, MISSIONS, VISIONS, PLANS, etc. and extracting these from the narrative, OR
2. Reverse mapping—Starting with outcomes, ask “how to” in order to generate the activities which produce them, OR
25
p ,3. Forward mapping—Starting with activities,
ask “so what” in order to generate the outcomes that are expected to result
Then…Do Some Sequencing…
Divide the activities into 2 or more l b d th i l i lcolumns based on their logical
sequence. Which activities have to occur before other activities can occur?
26
Do same with the outcomes. Which outcomes have to occur before other outcomes can occur?
![Page 14: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/14.jpg)
14
Infrastructure
Year 1 Years 2-4 Project End
Increased skill-sets of SHD
Implemented strategic plans
Increased diversity of IVP funding and
• Formative •Formative
and resources
•Summative
![Page 15: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/15.jpg)
15
Evaluation
Year 1 Years 2-4 Project End
Established evaluation & monitoring outcomes
Self-evaluation of activities
Achievement of 4 & 5 year goals
• Formative •Formative•Summative•Health Impact
Strategies
Year 1 Years 2-4 Project End
Established MOUs for policy & intervention
Implemented 4 priority topic t t i
Evidence influenced strategies
intervention plans
• Formative
strategies
•Formative•Summative•Health Impact
![Page 16: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/16.jpg)
16
Collaboration
Year 1 Years 2-4 Project End
Increased public awareness of injury &
Enhanced partnerships
Increased political will for IVP issuesviolence
• Formative •Formative
issues
•Summative
Surveillance
Year 1 Years 2-4 Project End
Implemented surveillance standards
Strengthened injury & violence surveillance
Effective use of state injury & violence data
• Formative
surveillance
•Formative
data
•Summative
![Page 17: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/17.jpg)
17
For Planning and Evaluation “Causal” Arrows Can Help
Not a different logic model, but same elements in different formatArrows can go from:
Activities to other activities Which activities feed which other activities?
Activities to outcomes
33
Activities to outcomesWhich activities produce which intended outcomes?
Early effects/outcomes to later onesWhich early outcomes produce which later outcomes
![Page 18: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/18.jpg)
18
Elaborating Your Program Logic
Common TermsMediatorsOutputsInputsModerators
BUT..
35
Not all models need to have (all of) themIf you use them, use them well
Integrating Program Planning and Evaluation
Elaborating Our ProgramElaborating Our Program “Theory”—Mediators
![Page 19: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/19.jpg)
19
Filling in the Blanks….
37 37
Inputs Activities Outcomes
Implicit Logic Model
Conduct trainingDo career leadership
development
Competent, sustainable,
diverse workforce
Improved health
outcomesDo Strategic workforce planning
Evidence Base
38
planning
![Page 20: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/20.jpg)
20
Implicit Logic Model
Inputs Activities Outcomes
Conduct trainingDo career leadership
development
Workforceis
CompetentSustainable
Diverse
Improved health
outcomesDo Strategic
kf
Evidence Base
p
39
workforce planning
ConductWorkforce
is:
Inputs Activities Outcomes
Implicit Logic Model—Mediators
Conduct training
Do career leadership
development
is:
Competent
SustainableImproved
health outcomes
Do Strategic workforce
Evidence Base
Programs are more effective
Continuity in r’ships
and approach
40
Diverseworkforce planning Clients
access and adhere
![Page 21: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/21.jpg)
21
Infrastructure
Year 1 Years 2-4 Project End
Increased skill-sets of SHD
Implemented strategic plans
Increased diversity of IVP funding and
• Formative •Formative
and resources
•Summative
Evaluation
Year 1 Years 2-4 Project End
Established evaluation & monitoring outcomes
Self-evaluation of activities
Achievement of 4 & 5 year goals
• Formative •Formative•Summative•Health Impact
![Page 22: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/22.jpg)
22
Integrating Program Planning and Evaluation
Elaborating Our ProgramElaborating Our Program “Theory”—Outputs, Inputs, and Moderators
Tangible products of activities
Activities Inputs Outputs
Intermediate Effects/
Outcomes
Short-term
Effects/ Outcomes
Long-term Effects/
Outcomes
44
AssumptionsContext/External Factors
![Page 23: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/23.jpg)
23
Upgrading Your Outputs—How Logic Models Help
OutreachOutreach
Behavior
Screening
Train in Self-
45
ChangeImproved Health Outcomes
ID People with Condition
Management
Refer for Medical Treatment
Medical Management
Traditional Outputs—Typical Screening Program
Screening: Pool (#) of screenedScreening: Pool (#) of screened kidsReferrals: (#) referrals to medical treatmentTraining: Pool (#) of families trained
46
Training: Pool (#) of families trained
![Page 24: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/24.jpg)
24
The Plot Thickens
Screening soScreening so good it will lead to…
Behavior Change
ID People with Condition
Training in Self-Management so good it will lead
4747
to…
Referral for Medical Treatment so good it will lead to…
Quality Medical Management
“Upgraded” Outputs: More than Simple Counts
Screening: Pool (#) of screened kids (meeting likely risk profile)Training: Pool (#) of clients trained using culturally-competent curriculum and with appropriate supports Referrals: Pool(#) of referrals to
4848
Referrals: Pool(#) of referrals to (qualified or willing) medical treatment providers
![Page 25: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/25.jpg)
25
Inputs and Moderators
I h f “killIn search of “killer assumptions”
50
![Page 26: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/26.jpg)
26
Resource “platform” for the program
Activities Inputs Outputs
Intermediate Effects/
Outcomes
Short-term
Effects/ Outcomes
Long-term Effects/
Outcomes
51
Context/External Factors
Moderators: Contextual factors th t ill f ilit t
Activities Inputs Outputs
Intermediate Effects/
Outcomes
Short-term
Effects/ Outcomes
Long-term Effects/
Outcomes
that will facilitate or hinder getting our outcomes
52
Context/External Factors
![Page 27: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/27.jpg)
27
Understanding Our Program Logic—How Inputs and Moderators Help
From this
IF THEN
From this…
To this…IF
AND
THEN
Basic CPPW Logic Model—Where Inputs and Moderators Live
Activities and
Supports
Changes in
• Policies• Systems• Environments
Changes in
• Physical activity• Nutrition• Tobacco use• Exposure to tobacco smoke
Reductions in
• Obesity• Smoking rates• Morbidity• Mortalitynp
uts
tobacco smoke• Use of quit lines
MortalityI
Moderators
![Page 28: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/28.jpg)
28
Sample Inputs—Screening Program
FundsTrained staffLegal authority to screenRelationships for med treatment
55
![Page 29: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/29.jpg)
29
Moderators—Four Types
P liti lPoliticalEconomicSocialTechnological
57
Technological
Inputs and Moderators—What Do We Learn
ProspectivelyRedesignWork-arounds
Retrospectively
58
Run toward the killer assumption
![Page 30: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/30.jpg)
30
Note!
Logic Models make the program theory clear,
not true!
59
Integrating Program Planning and Evaluation
Putting Your Logic Model toPutting Your Logic Model to Use
![Page 31: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/31.jpg)
31
Framework forProgram Evaluation
“Program description” stage frames decisions about evaluation focus
61
Step 1. Which Stakeholders Matter Most?
Who is:Affected by the program?Involved in program operations?Intended users of evaluation findings?
Of these, who do we most need to:Enhance credibility?Implement program changes?Advocate for changes?Fund, authorize, expand program?
![Page 32: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/32.jpg)
32
Step 3. Key Domains in Eval FocusImplementation (Process)
Is program in place as intended? p g p
Effectiveness (Outcome)Is program achieving its intended short-, mid, and/or long-term effects/outcomes?
Efficiency yHow much “product” is produced for given level of inputs/resources?
Causal Attribution Is progress on outcomes due to your program?
Utility Questions
Purpose: Toward what end is thePurpose: Toward what end is the evaluation being conducted?User: Who wants the info and what are they interested in? U H ill th th i f ?Use: How will they use the info?
![Page 33: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/33.jpg)
33
(Some) Potential Purposes/Uses
Show accountabilityShow accountabilityTest program implementation“Continuous” program improvementIncrease the knowledge baseOther…Other…
(Some) Potential Purposes/Uses
Show accountabilityShow accountabilityTest program implementation“Continuous” program improvementIncrease the knowledge baseOther…Other…
![Page 34: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/34.jpg)
34
Logic Models and Strategy and Planning
By Clarifying Sequence of Activities y y g qOutcomes, Helps With…
Identifying/refining missionDeveloping consensus on key intended outcomes—the “staked claim”Defining goals and objectives
67
Defining goals and objectives, strategies and actions
Key Take-Aways1. It’s never about the model, it’s about
“understanding your program”2. You need this for “organizational learning”
and not just “accountability”3. Most/best benefits may be process use
benefits4. You know lots about your program even
68
before you draw your model5. A little bit of logic modeling goes a long
way. There’s a trade-off between “accuracy” and “utility”
![Page 35: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/35.jpg)
35
Where Next….
Identify evaluation questionsIdentify evaluation questionsDefine indicators and data sources for questionsAnalyze dataDraw conclusions and resultsTurn results into action
But…
Later Steps Informed by WorkLater Steps Informed by Work of Earliest Steps….
![Page 36: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/36.jpg)
36
Helpful Publications @ www.cdc.gov/eval
71 71
Helpful ResourcesNEW! Intro to Program Evaluation for PH Programs—A Self-Study Guide: http://www.cdc.gov/eval/whatsnew.htmLogic Model Sites
Innovation Network: http://www.innonet.org/Harvard Family Research Project: http://www.gse.harvard.edu/hfrp/University of Wisconsin-Extension: http://www.uwex.edu/ces/lmcourse/CDC/DASH:http://www.cdc.gov/healthyyouth/evaluation/resources.htm#4CDC/STD: http://www.cdc.gov/std/program/progeval/TOC-PGprogeval.htm
TextsKellogg Foundation Logic Model Development Guide:
72
Kellogg Foundation Logic Model Development Guide: www.wkkf.orgW.K. Kellogg Foundation Evaluation Resources: http://www.wkkf.org/programming/overview.aspx?CID=281Rogers et al. Program Theory in Evaluation. New Directions Series: Jossey-Bass, Fall 2000Chen, H. Theory-Driven Evaluations. Sage. 1990
![Page 37: Integrating ProgramIntegrating Program Planning and ... · You need this for “organizational learning” and not just “accountability” 3. Most/best benefits may be process use](https://reader035.vdocuments.us/reader035/viewer/2022071214/60436e6af00540673b6ba492/html5/thumbnails/37.jpg)
37
Community Tool Boxhttp://ctb.ku.edu
73 73