developing a common monitoring and evaluation (m&e) framework for the iasc guidelines on mental...
TRANSCRIPT
![Page 1: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/1.jpg)
Developing a common Monitoring and Evaluation (M&E) framework for the IASC
Guidelines on Mental Health and Psychosocial Support (MHPSS) in
emergency settings
Jura Augustinavicius, MSc MHS
Doctoral Student
Johns Hopkins Bloomberg School of Public Health
![Page 2: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/2.jpg)
Outline
• Key Monitoring and Evaluation (M&E) concepts• Main components of Logical Frameworks
(Logframes)• IASC• Need for a common M&E framework• Review
– Aims– Introduction of Draft I of the Common M&E Framework– Methods– Results
• Conclusions
![Page 3: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/3.jpg)
Key Concepts: M&E
Functions of M&E:
1. Process Evaluations: How well was the project implemented?
2. Monitoring outputs and outcomes: Did changes occur in the target population?
3. Impact evaluation: Can observed changes be attributed to the project or programme (i.e. cause and effect)?
![Page 4: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/4.jpg)
Key Concepts: Logframes
• A systematic tool for designing, planning, implementing, and monitoring and evaluating a project or programme– Can help organize thinking – Identify and assess risks inherent in project
design and implementation– Measure project progress– Develop consensus and communicate a
project’s intent and strategy
![Page 5: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/5.jpg)
PROJECT STRATEGY
GOAL: The overall good to which the programme contributes (and to which other factors and programmes may also contribute)
![Page 6: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/6.jpg)
PROJECT STRATEGY
GOAL: The overall good to which the programme contributes (and to which other factors and programmes may also contribute)
OUTCOMES: The specific parts of the overall goal that the project will achieve and that the project is held responsible for
![Page 7: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/7.jpg)
PROJECT STRATEGY
GOAL: The overall good to which the programme contributes (and to which other factors and programmes may also contribute)
OUTCOME: The specific part of the overall goal that the project will achieve and that the project is held responsible for
OUTPUTS: The project intervention; Direct and immediate products of activities
![Page 8: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/8.jpg)
PROJECT STRATEGY
GOAL: The overall good to which the programme contributes (and to which other factors and programmes may also contribute)
OUTCOME: The specific part of the overall goal that the project will achieve and that the project is held responsible for
OUTPUTS: The project intervention; Direct and immediate products of activities
ACTIVITIES: What people who are employed by the project do to assist the community
![Page 9: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/9.jpg)
PROJECT STRATEGY
GOAL: The overall good to which the programme contributes (and to which other factors and programmes may also contribute)
OUTCOME: The specific part of the overall goal that the project will achieve and that the project is held responsible for
OUTPUTS: The project intervention; Direct and immediate products of activities
ACTIVITIES: What people who are employed by the project do to assist the community
INPUTS: Major resources required to carry out activities
![Page 10: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/10.jpg)
PROJECT STRATEGY
ASSUMPTIONS: Variables that are outside of the control of the project but that affect the success of the project
• Use “if, then” logic
• Should be generated based on past experience with similar programs
GOAL: The overall good to which the programme contributes (and to which other factors and programmes may also contribute)
OUTCOME: The specific part of the overall goal that the project will achieve and that the project is held responsible for
OUTPUTS: The project intervention; Direct and immediate products of activities
ACTIVITIES: What people who are employed by the project do to assist the community
INPUTS: Major resources required to carry out activities
![Page 11: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/11.jpg)
PROJECT STRATEGY
INDICATORS: Used to measure progress towards achieving the project strategy
ASSUMPTIONS: Variables that are outside of the control of the project but that affect the success of the project
GOAL: The overall good to which the programme contributes (and to which other factors and programmes may also contribute)
OUTCOME: The specific part of the overall goal that the project will achieve and that the project is held responsible for
OUTPUTS: The project intervention; Direct and immediate products of activities
ACTIVITIES: What people who are employed by the project do to assist the community
INPUTS: Major resources required to carry out activities
![Page 12: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/12.jpg)
PROJECT STRATEGY
INDICATORS: Used to measure progress towards achieving the project strategy
MEANS OF VERIFICATION: Sources of indicator data
ASSUMPTIONS: Variables that are outside of the control of the project but that affect the success of the project
GOAL: The overall good to which the programme contributes (and to which other factors and programmes may also contribute)
OUTCOME: The specific part of the overall goal that the project will achieve and that the project is held responsible for
OUTPUTS: The project intervention; Direct and immediate products of activities
ACTIVITIES: What people who are employed by the project do to assist the community
INPUTS: Major resources required to carry out activities
![Page 13: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/13.jpg)
IASC
• IASC guidelines were developed in 2007
• Published consensus on MHPSS in emergencies
• IASC MHPSS Reference Group continues to meet to disseminate and implement the guidelines and to develop MHPSS products
• 30+ member organization
![Page 14: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/14.jpg)
A Common M&E Framework
• Systematic M&E can offer a means to understand the relative benefits of MHPSS interventions
• Consolidate goals, outcomes, and indicators for MHPSS activities across organizations
• Offers a common platform from which to advocate for prioritizing MHPSS in humanitarian settings
![Page 15: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/15.jpg)
Aim
To identify the most commonly applied strategies to monitor and evaluate MHPSS interventions in LMICs using a logframe approach
![Page 16: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/16.jpg)
Methods1. Review of Logframes and Theories of Change
Documents collected from IASC RG MHPSS member organizations across humanitarian sectors and anonymized
![Page 17: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/17.jpg)
Methods1. Review of Logframes and Theories of Change
Documents collected from IASC RG MHPSS member organizations across humanitarian sectors and anonymized
2. Narrative Review
EMBASE, PILOTS, PsycInfo, PubMed/MEDLINE, and WHO regional databases
![Page 18: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/18.jpg)
Methods1. Review of Logframes and Theories of Change
Documents collected from IASC RG MHPSS member organizations across humanitarian sectors and anonymized
2. Narrative Review
EMBASE, PILOTS, PsycInfo, PubMed/MEDLINE, and WHO regional databases
• Standardized instructions for double data extraction of: • Goals • Outcomes• Indicators• Means of Verification
• Top-down: Mapping outcomes and indicators to outcomes from the preliminary IASC RG framework
• Bottom-up: Qualitative (thematic) analysis to identify common themes at each logframe level
![Page 19: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/19.jpg)
Goal: Enhanced mental health and psychosocial wellbeing/reduced psychological and social suffering
Outcomes:
Community-focused (1) Emergency responses are safe, dignified, participatory, community owned, socially and culturally acceptable
(2) People are safe, protected and human rights violations are addressed
(3) Existing social, community, and family systems promote an environment for recovery and human development
Person-focused (4) Communities and families support people with mental health and psychosocial problems
(5) People with mental health and psychosocial problems access appropriate focused care
Underlying core principles: 1. Human rights and equity; 2. Participation; 3. Do no harm; 4. Integrated services and supports; 5. Building on available resources and capacities; 6. Multi-layered supports
Developed based on input from workshops in Sri Lanka, Lebanon, and Uganda
![Page 20: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/20.jpg)
Results: Logframe & TOC Review
39 Documents reviewed• Logframes
(84.6%)
![Page 21: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/21.jpg)
Results: Logframe & TOC Review
![Page 22: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/22.jpg)
Results: Narrative Review
88 Articles and Book Chapters• Studies published between
1992-2013
Study Population• Children and adolescents
(31%)• Adults (32%)• Combination of children
and adults (23%)• Unknown (14%)
![Page 23: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/23.jpg)
Results: Narrative Review
![Page 24: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/24.jpg)
Qualitative Analysis Results
• Theme 1: Increasing resilience, promoting social wellbeing, and preventing mental health and psychosocial problems
• Theme 2: Reducing ill mental health, psychosocial symptoms, and associated functional impairment through care
• Theme 3: Building capacity to identify, intervene on, and monitor MHPSS problems
• Theme 4: Promoting optimal human development within existing social systems
• Theme 5: Macro level goals that seek to build peace between groups after conflict and to address structural problems within societies
• Theme 6: Protection of vulnerable groups, such as women, children, the elderly, and people with disabilities
![Page 25: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/25.jpg)
Qualitative Analysis Results
• Theme 1: Increasing resilience, promoting social wellbeing, and preventing mental health and psychosocial problems
• Theme 2: Reducing ill mental health, psychosocial symptoms, and associated functional impairment through care
• Theme 3: Building capacity to identify, intervene on, and monitor MHPSS problems
• Theme 4: Promoting optimal human development within existing social systems
• Theme 5: Macro level goals that seek to build peace between groups after conflict and to address structural problems within societies
• Theme 6: Protection of vulnerable groups, such as women, children, the elderly, and people with disabilities
![Page 26: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/26.jpg)
Qualitative Analysis Results
• Theme 1: Increasing resilience, promoting social wellbeing, and preventing mental health and psychosocial problems
• Theme 2: Reducing ill mental health, psychosocial symptoms, and associated functional impairment through care
• Theme 3: Building capacity to identify, intervene on, and monitor MHPSS problems
• Theme 4: Promoting optimal human development within existing social systems
• Theme 5: Macro level goals that seek to build peace between groups after conflict and to address structural problems within societies
• Theme 6: Protection of vulnerable groups, such as women, children, the elderly, and people with disabilities
![Page 27: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/27.jpg)
Qualitative Analysis Results
• Theme 1: Increasing resilience, promoting social wellbeing, and preventing mental health and psychosocial problems
• Theme 2: Reducing ill mental health, psychosocial symptoms, and associated functional impairment through care
• Theme 3: Building capacity to identify, intervene on, and monitor MHPSS problems
• Theme 4: Promoting optimal human development within existing social systems
• Theme 5: Macro level goals that seek to build peace between groups after conflict and to address structural problems within societies
• Theme 6: Protection of vulnerable groups, such as women, children, the elderly, and people with disabilities
![Page 28: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/28.jpg)
Qualitative Analysis Results
• Theme 1: Increasing resilience, promoting social wellbeing, and preventing mental health and psychosocial problems
• Theme 2: Reducing ill mental health, psychosocial symptoms, and associated functional impairment through care
• Theme 3: Building capacity to identify, intervene on, and monitor MHPSS problems
• Theme 4: Promoting optimal human development within existing social systems
• Theme 5: Macro level goals that seek to build peace between groups after conflict and to address structural problems within societies
• Theme 6: Protection of vulnerable groups, such as women, children, the elderly, and people with disabilities
![Page 29: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/29.jpg)
Qualitative Analysis Results
• Theme 1: Increasing resilience, promoting social wellbeing, and preventing mental health and psychosocial problems
• Theme 2: Reducing ill mental health, psychosocial symptoms, and associated functional impairment through care
• Theme 3: Building capacity to identify, intervene on, and monitor MHPSS problems
• Theme 4: Promoting optimal human development within existing social systems
• Theme 5: Macro level goals that seek to build peace between groups after conflict and to address structural problems within societies
• Theme 6: Protection of vulnerable groups, such as women, children, the elderly, and people with disabilities
![Page 30: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/30.jpg)
Conclusions
• The current draft of the common MHPSS M&E framework can be applied to existing documents and peer-reviewed literature
• Confusion with regard to defining goal and outcomes
• Although psychometrically sounds means of verification are published in the peer-reviewed literature, these are not commonly applied to logframes or theories of change
![Page 31: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/31.jpg)
IASC MHPSS Reference Group
Wietse Tol
Ananda Galappatti
Saji Thomas
Alison Schafer
Margriet Blaauw
Mark van Ommeren
Johns Hopkins Bloomberg School of Public Health
M. Claire Greene
Annisa Harsha
Megan Cherewick
Jillian Emerson
Katharine Shelley
Amanda Samson
Srihari Cattamanchi
Saumya Singh
Acknowledgements
With funding from UNICEF
![Page 32: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/32.jpg)
Additional Slides
![Page 33: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/33.jpg)
Key Concepts: M&E
Monitoring: A continuing function that aims primarily to provide the management and main stakeholders of an ongoing intervention with early indicators of progress, or lack thereof, in the achievement of results
![Page 34: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/34.jpg)
Key Concepts: M&E
Evaluation: A selective exercise that attempts to systematically and objectively assess progress towards and the achievement of a health outcome
![Page 35: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/35.jpg)
Key Concepts: M&EProgramme Monitoring Programme Evaluation
What is it?• Assessment of whether a
project is being implemented as planned
Assessment of whether:• The project achieved the
objectives/intended results• The project provided value
and made a difference
Why do it?
When is it done?
What is assessed?
![Page 36: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/36.jpg)
Key Concepts: M&EProgramme Monitoring Programme Evaluation
What is it?• Assessment of whether a
project is being implemented as planned
Assessment of whether:• The project achieved the
objectives/intended results• The project provided value
and made a difference
Why do it?• To determine what changes
are needing during the project• To decide what activities to
repeat in future projects
When is it done?
What is assessed?
![Page 37: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/37.jpg)
Key Concepts: M&EProgramme Monitoring Programme Evaluation
What is it?• Assessment of whether a
project is being implemented as planned
Assessment of whether:• The project achieved the
objectives/intended results• The project provided value
and made a difference
Why do it?• To determine what changes
are needing during the project• To decide what activities to
repeat in future projects
When is it done?
• During the life of the project• Frequently or continuously
• At the beginning and end of a project (occasionally at the midpoint)
What is assessed?
![Page 38: Developing a common Monitoring and Evaluation (M&E) framework for the IASC Guidelines on Mental Health and Psychosocial Support (MHPSS) in emergency settings](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649e915503460f94b95e8f/html5/thumbnails/38.jpg)
Key Concepts: M&EProgramme Monitoring Programme Evaluation
What is it?• Assessment of whether a
project is being implemented as planned
Assessment of whether:• The project achieved the
objectives/intended results• The project provided value
and made a difference
Why do it?• To determine what changes
are needing during the project• To decide what activities to
repeat in future projects
When is it done?
• During the life of the project• Frequently or continuously
• At the beginning and end of a project (occasionally at the midpoint)
What is assessed?
• Resource amounts and their distribution
• Delivery, distribution, and quantity of services
• Costs and expenditures
• Actual changes relative to intended changes
• Value of project activities and achievements