patricia rogers, rmit university, melbourne part 3 of 8 aea coffee break webinars 2013 frame what is...
TRANSCRIPT
Part 3 of 8 AEA Coffee Break Webinars 2013
Patricia Rogers, RMIT University, Melbourne
FRAME what is to be evaluated
Why do we need to start with a clear definition?
Source: Hobbies on a Budget / Flickr
Frame Decision Make Decision
Why do we need to start with a clear definition?
Source: Hobbies on a Budget / Flickr
Frame Decision Make Decision
Design Evaluation
Why do we need to start with a clear definition?
Source: Hobbies on a Budget / Flickr
Frame Decision Make Decision
Frame Evaluation Design Evaluation
Four evaluation tasks in FRAMING
Identify primary intended
users
Decide purpose(s
) (intended
uses)
Specify key
evaluation
questions
Determine what
‘success’ looks like
Possible primary intended users
Families
Teachers
Principal
School Council
Education Departmen
tOther
schools
Image source: IN 157s01 – Curt Carnemark/ World Bank
Formative – improve it
Summative – continue or stop
itBroader evidence
base
Purposes (intended uses)
Image source: CK-CO138 - Charlotte Kesl / World Bank
Lobby and advocate
Interconnection between intended users and intended uses
Identify primary intended users
Decide purpose(s)
(intended uses)
FRAME
Descriptive:How many children
attend? What learning tools are
used? Has learning improved?
Key evaluation questions
Image source: ML030S09 - Curt Carnemark / World Bank
Synthesis:Has the program been a
success? Is it Value For Money
compared to alternatives?
Key evaluation questions
MANAGE
DEFINE
FRAME
DESCRIBE
UNDERSTAND CAUSES
SYNTHESIZEREPORT & SUPPORT USE
Descriptive Questions-What were the activities, changes, context?
Causal questions –What caused orcontributed to the identified changes?
Synthesis questions –Overall was it good? Value for money?
Action questions-What should we do?
Options for answering different types of questions
Processes:Are students
fairly treated?
What does success look like?
Image source: LQ-CN7499 World Bank
What does success look like?
Distribution of costs and benefits:
Who benefits from the program? Whose needs are met? Who is disadvantaged by
it?
What does success look like?
Criteria and standards:
Reading levels – better than before,
or better than national average, or national benchmark?
You can read about this case here:
http://betterevaluation.org/blog/outcome_harvesting_bionet
Patricia RogersRMIT University
Further resources
http://betterevaluation.org/plan/framefacebook.com/betterevaluation@bettereval