Designing an M&E System for Market Engagement at CARE
Christian Pennotti
Technical Advisor, Learning & Impact
Economic Development Unit
CARE’s M&E Approach – Questions for Consideration
How can we provide guidance that ensures M&E becomes an integral part of regular project processes?
How can we provide guidance that enable teams to design M&E that effectively responds to value chain dynamics?
How can we combine an increased desire to be held directly accountable to our impact group members while maintaining reasonable cost structures for M&E?
Core Principles
1. Base M&E on a clear and well-documented causal model
2. Support project management and accountability to target beneficiaries
3. Emphasize leading outcomes in monitoring and lagging outcomes in evaluation
4. Incorporate learning loops that leverage both tacit and explicit knowledge from multiple stakeholders
5. Be flexible and able to adapt to project evolving project priorities
6. Be feasible to implement with resources and capacity
Steps to Designing the M&E System
Core Principles
1. Base M&E on a clear and well-documented causal model
2. Support project management and accountability to target beneficiaries
3. Emphasize leading outcomes in monitoring and lagging outcomes in
evaluation
4. Incorporate learning loops that leverage both tacit and explicit knowledge from
multiple stakeholders
5. Be flexible and able to adapt to project evolving project priorities
6. Be feasible to implement with resources and capacity
Step 1M&E System Client
Mapping
Step 10Implement M&E
System, Review and Adjust
Step 9Provide Training to All
Personnel Responsible for M&E
Step 8Develop Data Management
Protocols
Step 7Ensure Incentive
Alignment
Step 6Design Data Analysis
and Use Plan
Step 5Identify Data
Collection Tools
Step 4Identify Indicators
Step 3Assess M&E
Resources & Capacity
Step 2Develop / Refine
Causal Model
1. M&E System Client Assessment
Power / Relevance Mapping
Relevance
Application • Participatory exercise
• Group brainstorm
• Small groups discuss and place stakeholders on the matrix
• Facilitated discussion about variance until final set of clients to be “Managed Closely” defined.
• Leads to discussion on client needs and expectations
2. Value Chain Initiative Causal Model
text
Enterprise
Results
Women’s Empowerment
Results
Sector
Results
Direct / Enterprise
Results
Women’s Empowerment
Results
Indirect / Sector
Results
Sustainable Household Livelihoods & Women’s Empowerment
Impacts Impacts
Activities & Outputs
Activities & Outputs
Leading Outcomes:Knowledge, Attitudes, Practices
Lagging Outcomes: Countable,
Tangible Chnages
Assumptions
Application • Participatory exercise
• Break the causal model into pieces
• Small groups discuss and place results on the matrix
• Facilitated discussion about variance until final agreement reached on the project’s fundamental logic
• Leads to clarity on ‘killer assumptions’ as well as indicator needs
2. Example from Peru
Women’s Emp. Enterprise Sector
Leading
Lagging
Activities
Impacts
4. Indicator Selection
Two types of indicators
Those tracked through routine measurement – traditional M&E
Those tracked through routine observation – leveraging staff observation
4. Indicator Selection – Identifying Killer Assumptions
Application • Participatory exercise
• Pull out the ‘lines’ connecting causal pathways in the causal model
• Blind ranking of assumptions with the greatest risk and greatest potential reward
• Definition of ‘indicators’ to monitor validity of prioritized ‘killer assumptions’ via observation
Step 5: Select Tools
Indicators to be tracked through routine measurement Traditional M&E tools (FGDs, surveys, etc)
Indicators to be tracked through routine observation KM-oriented tools (After Action Reviews,
routine meetings, common learning agenda, observation cards, etc)
Step 6: Designing Data Analysis & Use
Indicators for Routine Measurement Highly related to traditional practices but
with greater focus on leveraging data for multiple M&E System Clients
Indicators for Routine Observation Focus on KM tools, reflective practice and
increased opportunities for simple ranking and benchmarking performance across the project
Step 7: Ensure Incentive Alignment
To be developed
Focus on ensuring data quality and ability to improve decision making
Leverage examples raised in the e-consultation like HR integration, monetary incentives, peer pressure
Need to define limitations – not everything can be solved by the M&E system!
Illustrations of Routine Observation Integration
Zambia ADAPT Project Staff-defined monthly learning themes, consolidated
summaries Observation-based agro-dealer categorization tool
Bangladesh SDVC Project Routine, focused meetings at all levels of the project Participant-reporting categorization tool Regional performance scorecards for field managers
Ethiopia PSNP Plus Initiating geographically rotating technical working group
meetings
Moving Ahead
Developing data analysis / use guidelines
Developing incentives module
Target guide release – Fall, 2011
Ongoing application, revision and building use case library
Collaboration via GROOVE & MaFI to refine, apply, advance