arizona state university college of nursing and health innovation [email protected] linda k...
TRANSCRIPT
Arizona State University College of Nursing and Health [email protected]
Linda K Larkey, PhDScottsdale Healthcare Professor of Biobehavioral Oncology Research
overview
evaluation paradigms
what is an “evaluation theory”?
what does a theory buy you?
how do you build theoretical constructs into evaluation?
practice
positivist approach
assumes there are “objective”, observable and measurable aspects of a program
preference for predominantly quantitative evidence needs assessment assessment of program theory assessment of program process impact assessment efficiency assessment
(Rossi, Lipsey and Freeman, 2004)
three basic paradigms
positivist
Interpretive
critical-emancipatory
you don’t have to join a club--- feel free to blend
Potter, C. (2006). Program Evaluation. In M. Terre Blanche, K. Durrheim & D. Painter (Eds.), Research in practice: Applied methods for the social sciences (2nd ed.)
interpretive approaches
purports it is essential that the evaluator develops an understanding of the perspective, experiences, needs and expectations of all stakeholders
key words, “understanding” “meaning” “rich description”
considered crucial before one is able to make judgments about the merit or value of a program.
The evaluator’s contact with the program is important, over time observation Interviews focus groups
critical-emancipatory approaches based on action research for the
purposes of social transformation
particularly useful in developing countries or underserved populations
concerns: empowering populations to create change from within, providing “voice”
so what?
it matters whether you think your goal is to achieve some measurable state or reality…
or, if you think that the process along the way and people’s opinions and perceptions are most important
or, if you see the program goals to empower, raise consciousness, give voice
What is the result expected? Typical logic: Process: We will reach X number of
female, single heads of household with food stamp information by….
Short-term outcome: We will distribute $X within Maricopa County by…
Longer-term outcome: We will increase the use of food stamps by $X
Impact: We will reduce food shortage by X%
what “causes” the desired end result? how would we… reduce food instability in our state
reduce flu epidemics
increase number of servings of F & V eaten by school age children in school lunch
increase cancer screening rates what else?
Pick one “impact” level goal What causes it? (Needs assessment,
what is the problem, what contributes?)
What do you think might fix it? One thing or many?
How would you know if it were fixed?
That is your Evaluation Theory Once you name the causes, purported
improvement strategies, and expected outcome, you have a LOGIC MODEL
Your construal of the problem (antecedents) is YOUR THEORY
This theory, PLUS your assumptions about what is important to change to get a result is your EVALUATION THEORY
Simpler “Logic Model”: ATM
Low rates of colorectal
cancer screening
Patients can’t access
or are discouraged
from accessing services
Long Waits for services
Facilities Lacking
Not enough trained providers
Lack of funding for uninsured
(screening/ treatment)
Inadequate Lobbying
(for policy and/or funding) Existing health
care plans inadequate
Cultural factors
Providers not making referralsProviders not clear
on guidelines (who/what/when/ri
sk)
Patients not demanding screening
Patients not aware of need/importance/valu
e of screening
New and complicated
message(not your average bumper sticker)
Lack of office reminder and/or flagging systems
Conflicting research evidence, that
screening saves lives, or best method
Uncomfortable topic (distaste for “things
scatological”)
Inappropriate Utilization
Competing priorities during patient/provider appointment: too little
time
Mixed/unclear financial impact (will referred
patients return? reimbursement?)
Such complex map of problems… what’s the solution?
Research Project DevelopmentAnd Evaluation
State/County DataPublic Messaging
Capacity for DeliveryCommunity-based Organizations
VolunteersCommunity Visibility
Legislative/PolicyDialogue Leadership
AZCC
(academic)
ADHS
(public
health)
ACS
(private)
Health Services Delivery
Figure 2. Transdisciplinary Model Approach for Increasing CRC Screening in Arizona
building your own evaluation plan… share example of one of your programs map “antecedent conditions”– what are
the causes? (theory of the problem) so, if those are the causes, what could
fix this problem? (theory of solution) what is the final outcome expected? then…
measure what along the way? At the end? things you can count? meaning/perception? what does the community say? Remember
the “critical” view
holding your vision while negotiating the terrain…