developmental evaluation and social innovation michael quinn …€¦ · and social innovation...
TRANSCRIPT
DevelopmentalEvaluationandSocialInnovationMichaelQuinnPatton
SymposiumonDec10,2017inTokyoEvaluationthatSupportsSocialInnovation– NewHorizonbroughtbyDevelopmentalEvaluation
Organizedby GraduateSchoolofGovernanceStudies,MeijiUniversityCo-organizedbyCSONetworkJapanInstituteofProgramEvaluation,MeijiUniversitywiththesupportofJapanEvaluationSociety
2
Evidence-based Practice
Evaluation grew up in the “projects” testing models under a theory of change that pilot testing would lead to proven models that could be disseminated and taken to scale:
The search for best practicesand evidenced-based practices
3
OriginalPrimaryOptions
Formativeand
SummativeEvaluation
(Mid-termandEnd-of-ProjectReviews)
4
EvaluationTraditionalEvaluating…• Projects&Programs• LogicModels• TheoriesofChange• SMARTGoals• Implementation• Outcomes&ImpactsGenerating...• Judgments,Lessons• Recommendations
InnovativeNewDirections:Evaluating…
• Missionfulfillment• Strategy• Advocacycampaigns• Policychange• SystemsChange• Complexdynamicinterventions
5
Innovations&Challenges:Evaluating…
• Communityimpacts• Regionalinitiatives• Environmentalecosystemsustainability• Networksandcollaborations• Leadership• Inclusivenessanddiversity• Innovation• Collectiveimpact• Scaling
6
Principles
INNOVATION
8
DevelopmentalEvaluationDefined
Purpose:Developmentalevaluation(DE)informsandsupportsinnovativeandadaptivedevelopment incomplex
dynamic environments.
DEbringstoinnovationandadaptationtheprocessesofaskingevaluative questions,applyingevaluationlogic,andgatheringandreportingevaluativedatatosupportproject,program,product, and/ororganizational development withtimelyfeedback.
KeyDECharacteristics• Focusondevelopment (versusimprovement,accountabilityorsummativejudgment)
• Takesplaceincomplexdynamicenvironments• Feedbackisrapid(asrealtimeaspossible).• Theevaluatorworkscollaboratively withsocialinnovatorstoconceptualize,designandtestnewapproachesinalong-term,on-goingprocessof adaptation,intentionalchange,and development.
9
KeyDECharacteristics• TheDEevaluatorcanbepartoftheinterventionteam.
• Theevaluator'sprimaryfunctionsaretoelucidatetheinnovationandadaptationprocesses,tracktheirimplicationsandresults,andfacilitateongoing,real-time,data-baseddecision-makinginthedevelopmentalprocess.
• DEbecomespartoftheintervention.
10
Complexityconcepts&Evaluation
• Emergence:Self-organizing,Attractors•Nonlinear:Smallactionscanhavelargereactions. “TheButterflyWingsMetaphor
•Uncertainty;unpredictable;uncontrollable;unanticipatedconsequences
•Coevolution:Processuses;interdependence•Adaptation:Changingcontext
11
12
Purposes
Purpose Challenge Implications
1 Ongoing development
Implemented in complex & dynamic environment
No intention of becoming fixed;identifies effective principles
2 Adapting effective principles to new contexts
Innovative initiatives:Develop ‘their own’ version
Knowledge disseminated;sensitivity to context, capabilities & priorities
FundamentalIssue:HowtheWorldIsChanged
Top-downscalingof“provenmodels”withFidelityEvaluation
versusBottoms-upadaptivemanagement
andDevelopmentalEvaluation
13
Models vs. Principles
Identifying proven principles for adaptive management
(bottoms-up approach)versus
Identifying and disseminating proven models
(top down approach)14
15
16
5 Purposes
Purpose Challenge Implications
3 Pre-formative development of model
Dynamic situations require innovative solutions; Model does not exist
New models move into evaluation & others remain developmental
4 Major cross-scale developmental evaluation
Disrupt existing system; scale & systems change add levels of complexity
Adaptive cross-scale innovations assume complex dynamics: agility, responsiveness
17
5 Purposes
Purpose Challenge Implications
5. Developing a rapid response in turbulent environment:
Existing responses no longer effective as conditions change:
Planning, execution & evaluation occur simultaneously.
Rapid feedback, adaptation, & agile responses.
natural disasters,civil unrest,political turmoil,famine.../
turbulence,instability,dynamic,high uncertainty…/
DEPrinciples
1. Developmentalpurpose2. Evaluationrigor3. Utilizationfocus4. Innovationniche5.Complexityperspective6.Systemsthinking7.Co-creation8.Timelyfeedback
18
19
Globalproblemstranscendnationalandagencyboundaries
• Climatechange• Economicturbulence• Refugees• Virulentinfectiousdiseases• Dyingoceans• Globalcyber-terrorism• Internationaldrugcartels• Humantrafficking• Weaponstrafficking• Povertyandinequality• Multi-nationalcorporatecollusion
Innovationto AddressGlobalProblems
ØThedefinitionsoftheproblemsaredisputedØThe“facts”areamatterofintensedebateØPoliticsandspecialinterestsdominate:
-- nationalinterests-- multi-nationalcorporateinterests-- agencyagendas-- competitionforresources
ØThestakesarehugeØInnovationiscrucialØDEisappropriate
23
TRANSFORMATION
GlobalSystemsAnalysisSkills:InterconnectedSDGs
Two-EyedSeeing
TheGuidingPrinciplebroughtintotheIntegrativeScienceco-learningjourneybyMi'kmaw ElderAlbertMarshall.
Etuaptmumk istheMi'kmaw wordforTwo-EyedSeeing.
GlobalKnowledge
Local&indigenousknowledge
DE Interdependence
28