humanitarian performance monitoring toolkitunicefinemergencies.com/downloads/eresource/docs...the...
TRANSCRIPT
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
1
Humanitarian Performance Monitoring Toolkit See UNICEF Intranet http://intranet.unicef.org/emops/emopssite.nsf/root/PageCCCPM1
Executive Summary
Background
The global humanitarian context is continually evolving with increasing frequency of
disasters and continuing complex emergencies. At the same time there is an increasing
examination of the effectiveness of the international humanitarian system and continued high
expectations for improved coordination and greater accountability, including to affected
populations. In 2010, UNICEF further reinforced its commitment to its humanitarian
mandate with the release of the revised Core Commitments for Children in Humanitarian
Action (CCCs), the organization’s central policy on humanitarian action, which now brings a
stronger results focus to UNICEF humanitarian work and aligns UNICEF commitments to
global standards such as SPHERE and Education in Emergencies standards (INEE).
Building on existing experience of UNICEF and partners, this Humanitarian Performance
Monitoring (PM) Toolkit takes this further with a view to strengthening UNICEF performance
monitoring in humanitarian situations. As the UNICEF CCCs can only be realized through
close collaboration with UNICEF implementing partners, the Toolkit includes elements that
are intended for use with partners and can be promoted for use at cluster/sector level.
Why is this important?
In on-going humanitarian situations, strong performance monitoring is essential to support
effective response. Operating in humanitarian situations is associated with high risk:
programme design risks of ‘doing harm’ in complex contexts; implementation and financial
management risks due to a scale up in operations, often after a sudden increase in
resources; and reputational risk. More and more attention is being paid to the effectiveness
of humanitarian response. The best way to demonstrate effectiveness is through solid, high
frequency performance information, showing results for affected population. For more
information see Humanitarian PM Background Note.
What is different?
In on-going humanitarian situations, there is a higher frequency demand for information
about programme implementation and operational support. For UNICEF, there are three
overarching questions that need to be answered, both to effectively manage the response
and for accountability. These are: (1) How well are UNICEF and implementing partner
programmes contributing to strategic results and targets; (2) As country cluster/sector lead
agency (CLA) how well is UNICEF leading cluster/sector coordination; and (3) How well is
the clusters/sector UNICEF is leading delivering on results for the affected population? NB.
The country cluster/sector lead agency is not solely accountable for cluster/sector results (it
is a joint accountability for the whole cluster/sector). The cluster/sector lead agency role is to
ensure that systems are in place for the cluster/sector to monitor results and identify gaps.
To monitor at these three levels, UNICEF needs to go beyond input data, which is the
‘traditional’ method of performance monitoring in humanitarian situations, that may include
elements situation monitoring as well as budget monitoring. UNICEF can no longer solely
rely on low-frequency and high-cost outcome level surveys. UNICEF and partners need light
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
2
systems appropriate to context to get a combination of rough high-frequency data on outputs
and estimates of coverage.
What is the proposed approach to strengthen Humanitarian Performance Monitoring?
For UNICEF, humanitarian PM in this Toolkit is designed to feed into the Situation Report or
SitRep. The SitRep, now aligned to the CCCs, is proposed as the minimum monitoring and
reporting requirement for UNICEF COs in on-going humanitarian situations. With this,
UNICEF will be able to provide light but systematic results-focused reporting to the
Humanitarian Country Team as well as other partners and external audiences.
The Humanitarian PM Toolkit supports data collection feeding into the SitRep including:
• Improved and simplified programme implementation monitoring for UNICEF implementing partners – through PCA Monitoring and Reporting Addendum;
• Field Monitoring systems providing systematic light high coverage and high-frequency data on quality of programmes, bottlenecks in implementation, end use of supplies, negative impact and as a minimum feedback mechanism for affected populations;
• Light, high frequency assessment of progress in meeting Cluster/Sector Lead Agency accountabilities (where UNICEF is country cluster/sector lead agency).
• Periodic surveys (lower frequency outcome monitoring) where funding allows and ideally
as an Inter-Cluster/Sector exercise.
When should Humanitarian Performance Monitoring be Used?
The Humanitarian PM is designed to be flexible and adaptable to country contexts; UNICEF
CO and implementing partner capacity and to the resources available.
Supporting Cluster/Sector Performance Monitoring1 This Toolkit has been developed
for UNICEF and implementing partners however the approach and key tools are also
intended for sharing and use with cluster/sector partners. Tools can also be easily adapted
to support Cluster/Sector level monitoring and facilitate agreement on what to monitor within
Clusters/sector working groups.
Support for UNICEF CO Humanitarian PM. To support UNICEF COs in applying the
Humanitarian PM Toolkit a Community of Practice has been established that guarantees a
48 hour response to any queries. In addition a roster of trained internal and external
candidates suitable for surge capacity for Humanitarian PM. A programme performance
monitoring E-tool for use by UNICEF and implementing partners (and more widely where no
such national or Humanitarian Country Team platform is in place) has been developed2.
Feedback The Humanitarian PM Toolkit is based on focused pilot work in Pakistan and
South Sudan, and input from an internal reference group and key external partners.
UNICEF is committed to building on the Toolkit based on new experiences, adaptations and
refinements. Please provide feedback to Kate Alley ([email protected])
1 UNICEF’s Coordination accountability under Humanitarian Reform and the revised CCCs is for the
sector/cluster UNICEF leads at the country level. This means it is not contingent on any formal activation of the cluster approach in a country. UNICEF’s coordination accountability remains.
2 This will allow UNICEF and implementing partners to input, share and draw reports on a few common priority
indicators, and to feed into Humanitarian Country Team performance management.
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
3
Table of Contents
Executive Summary ............................................................................................................ 1
1. Introduction ..................................................................................................................... 4
Background – the Core Commitment for Children in Humanitarian Action (CCCs) ............ 4
What is different about Humanitarian PM? ........................................................................ 5
What is the proposed approach for Humanitarian PM? ...................................................... 6
How does Humanitarian PM link with other processes? .................................................... 6
2. Guide to Strengthening Humanitarian Performance Monitoring ............................... 10
Steps to Strengthen CO Humanitarian Performance Monitoring ...................................... 10
Accountabilities for CO Humanitarian Performance Monitoring ....................................... 16
Simplified Humanitarian Integrated Monitoring and Evaluation Plan (IMEP) and Results
Framework ...................................................................................................................... 20
3. Humanitarian Performance Monitoring Tools ............................................................. 30
Situation Report ............................................................................................................... 30
Indicator Guide ................................................................................................................ 40
PCA Monitoring and Reporting Addendum ...................................................................... 47
UNICEF PCA / Intervention Mapping ............................................................................... 59
Field Monitoring ............................................................................................................... 61
UNICEF Cluster / Sector Coordination Milestone Monitoring Tool ................................... 82
Cross-Sectoral Communication for Development (C4D) .................................................. 86
Outcome Surveys ............................................................................................................ 91
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
4
1. Introduction
Background – the Core Commitment for Children in Humanitarian
Action (CCCs) The purpose of the Humanitarian PM Toolkit is to support the effective implementation of
UNICEF Core Commitments for Children in Humanitarian Action (CCCs). This entails
strengthening UNICEF Humanitarian PM to monitor progress of results for children at a
greater frequency. It also entails equipping UNICEF to monitor progress for its country
cluster/sector Lead Agency responsibilities.
The CCCs are UNICEF’s core humanitarian policy to uphold the rights of children affected by
humanitarian crisis. They are a framework for humanitarian action, around which UNICEF
seeks to engage with partners. The intent of the updated CCCs remains to promote
predictable, effective and timely collective humanitarian action, putting forth clearly the results
where UNICEF can best contribute. Initially issued in 2000 and revised in 2004, the current
revision of the CCCs brings policy up to date with changes in global context for humanitarian
action, including new evidence and best practices, as well as humanitarian reform, in
particular the Cluster Approach. The CCCs include specific commitments to establish and
support coordination mechanisms (relevant to UNICEF’s role as cluster lead or partner), with
interagency assessments and strategies being critical parts. The commitments combine
cluster / sectoral, operational, and cross-cutting issues. The CCCs also bring a stronger
results focus to UNICEF humanitarian work while at the same time aligning UNICEF
commitments to global standards such as SPHERE Humanitarian Charter and Minimum
Standards in Humanitarian Response and the InterAgency Network for Education in
Emergencies (INEE) Minimum Standards in Education. See more…
The Cluster Approach, introduced as one of the pillars of Humanitarian Reform at the end of
2006, is intended is to strengthen system-wide preparedness and technical capacity to
respond to humanitarian emergencies by ensuring that predictable leadership in the main
sectors leads to predictable humanitarian response. UNICEF is Global Cluster Lead Agency
in Nutrition and WASH, co-lead in Education (with Save the Children), focal point agency for
the child protection and gender-based violence ‘areas of responsibility’ in the broader
protection cluster; and co-chair of the cross-cutting IASC Reference Group on Mental Health
and Psychosocial Support in Emergency Settings. UNICEF’s role at the country level may
differ across countries since country-specific arrangements are determined based on
agencies’ in-country strengths and contextual factors. The role of cluster/sector leads at the
country level is to facilitate a process aimed at ensuring well-coordinated and effective
humanitarian responses in the sector or area of activity concerned. Specific responsibilities of
cluster/sector leads at the country level are detailed in the Terms of Reference for
Cluster/Sector Lead at Country Level and include performance management.
For further background see Background Note for CMT
Why higher frequency Humanitarian PM is important.
• It allows UNICEF and implementing partners results for children to be regularly
monitored and tracked
• It helps manage risks: programme design risks of ‘doing harm’ in complex contexts and
rapid scale-up; financial and implementation management risks when there is sudden
increase in resources; reputational risks where UNICEF is unable to say what its
progress is
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
5
• The international community and especially donors are demanding increased
accountability and demonstration of results. Evaluations continue to show humanitarian
PM as a weakness for both the wider humanitarian system and for UNICEF specifically.
• For UNICEF, Humanitarian PM is a CO accountability as detailed in the CCCs
What is different about Humanitarian PM? In on-going humanitarian situations, there is a higher frequency demand for information
about programme implementation and operational support. For UNICEF, there are three
overarching questions that need to be answered, both to effectively manage the response
and for accountability. These are: (1) How well are UNICEF and implementing partner
programmes contributing to strategic results and targets; (2) As country cluster/sector lead
agency (CLA) how well is UNICEF leading cluster/sector coordination; and (3) How well is
the clusters/sector UNICEF is leading delivering on results for the affected population? NB.
The country cluster/sector lead agency is not solely accountable for all cluster/sector results
(it is a joint accountability for the whole cluster/sector), the cluster/sector lead agency is
accountable to ensure that systems are in place for the cluster/sector to monitor results and
identify gaps.
To monitor at these three levels, UNICEF needs to go beyond input data, which is the
‘traditional’ method of performance monitoring in humanitarian situations, that may include
elements situation monitoring as well as budget monitoring. UNICEF can no longer solely
rely on low-frequency and high-cost outcome level surveys. UNICEF and partners need light
systems appropriate to context to get a combination of rough high-frequency data on outputs
and estimates of coverage.
In humanitarian situations, UNICEF and implementing partners need light monitoring
systems that are appropriate to the context to provide ‘good enough’ high-frequency data on
outputs that allow estimates of programme coverage. This is also needed at the level of
cluster/sector coordination. Without this the gap in coverage will be unknown for too long –
adversely affecting effective response to the population (and UNICEF reputation). In
addition, UNICEF and implementing partners need similar high-frequency data on quality of
programme implementation through Field Monitoring visits. As the scale of the response
increases, so must the coverage of Field Monitoring to give greater confidence in rough
coverage data, to identify bottlenecks and to allow feedback from affected populations.
What is the objective of Humanitarian PM?
The objective of Humanitarian PM is to support UNICEF CO management in assessing
performance through;
• Contributing to improve predictable, effective and timely humanitarian action of UNICEF
and it partners, both UNICEF implementing partners and cluster/sector partners.
• Strengthening (wherever possible) the links between national government frameworks,
UNICEF humanitarian action programmes and UNICEF regular programmes.
• To reinforce accountability at UNICEF Headquarters, Regional and Country Offices in
line with the CCCs, within the broader framework of UNICEF’s global Performance
Management.
When should Humanitarian Performance Monitoring be Used?
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
6
What is the proposed approach for Humanitarian PM?
Humanitarian PM builds on the existing UNICEF CO Situation Report (SitRep) an
established reporting mechanism in humanitarian situations. The SitRep is the proposed
minimum reporting requirement for all UNICEF COs in Level II and III emergencies. It has
been re-designed to serve as a CO management tool (or ‘overview dashboard’) with key
overview data for CO senior management to assess the progress of the response.
The Humanitarian PM Toolkit provides guidance and tools to support data collection to feed
into the SitRep. What is appropriate to each context will vary, starting with the most basic
and extending to higher levels of investment:
• Low Cost Option: Improved and simplified programme implementation monitoring through selection of priority high frequency indicators for inclusion in cooperation agreements (through the PCA Monitoring and Reporting Addendum)
• Medium Cost Option: Field Monitoring providing systematic light high coverage and high-frequency data on quality of programmes, bottlenecks in implementation, end use of supplies and negative impacts as well as a minimum feedback mechanism for affected populations;
• Low Cost Option: Assessment of progress in meeting country Cluster/Sector Lead Agency accountabilities (where UNICEF is country cluster lead agency).
• High Cost Option: Periodic surveys (lower frequency outcome monitoring) where
funding allows and ideally as an Inter-Cluster exercise.
The Humanitarian PM Toolkit focuses primarily on performance monitoring of UNICEF
support to service delivery and cluster/sector coordination (where UNICEF is country
cluster/sector lead agency). Where COs humanitarian response role is primarily focused on
capacity development and advocacy strategies with national partners, the focus of
performance monitoring efforts will be more in line with mainstream UNICEF CO
performance monitoring approaches. In such contexts some elements of the Toolkit may be
useful to promote for national partners use.
How does Humanitarian PM link with other processes?
• The Humanitarian PM Toolkit focuses specifically on Performance Monitoring.
• It does not provide detailed guidance or tools on initial Needs Assessment or
humanitarian evaluation. Both are covered in Inter-Agency resources.
Needs Assessments
• Accountability for Sudden Onset Response coordination and overall information
management lies with OCHA (in coordination with the national government)
• The scope of OCHA’s role will be a factor of its pre emergency presence, resources
available and role of national government.
• The UNICEF CO should determine the UNICEF role in Rapid Needs Assessments and
plan accordingly, see Steps to Strengthen CO Humanitarian PM
Situation Monitoring
• Options for Outcome Surveys are contained within the Humanitarian PM toolkit and guide
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
7
• Low frequency survey indicators contained in the Indicator Guide tool are mapped onto MICS indicators where relevant
Real Time Evaluations
• Real-time evaluations often take place at the Inter-Agency level.
• UNICEF will likely be involved as a key informant. UNICEF CO involvement in the
evaluation method itself will be determined on a case-by-case basis
Learning/Accountability Processes
• The following diagram shows the Humanitarian PM Toolkit in relation to wider Learning
and Accountability processes
• See the Simplified Humanitarian IMEP for additional details and resources
Figure: Humanitarian Learning/Accountability Options
Early Warning / Early Action Platform
• Note that Humanitarian PM also includes performance monitoring of preparedness
measures.
• For UNICEF COs, this is covered through the regular updates to the Early Warning-Early
Action System
• The overview tool Steps to Strengthen CO Humanitarian PM contains some light,
specific, preparedness steps that can be considered for Humanitarian PM.
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
8
VISION
• Programme-related data in humanitarian PM will be at a higher level of detail than
Programme Component Results (PCRs) and Intermediate Results (IRs) in the Revised
Results Structure (RRS) (e.g. an IR will at best be equivalent to the macro level nutrition
CCC strategic result); however, programme-related data in humanitarian PM can feed
into VISION performance data.
• Humanitarian PM will draw on operations performance data from PROMS/VISION (e.g.
financial data) as well as from other stand-alone data management ‘systems’ such DHR
tools on surge capacity management.
• In Humanitarian PM, programme-related data must be outside VISION as this must feed
into and be shared with national partners, NGO partners (with whom we have
cooperation agreements), wider Cluster members and HCT
• From 2012 (after VISION is introduced), it is proposed that the current microsoft word
SitRep template could be replaced with an electronic SitRep tool for greater ease and
efficiency in reporting.
• This would draw from the country level electronic humanitarian partner reporting format
(to be developed Q3/42011) as well as VISION, while allowing space for COs to add in
written analysis as in the current template.
• The E-SitRep would handle production of internal and external versions of the report as
well as allow storage of data (monthly time series) and would feed into aggregation of
key data at global level.
Key Assessment Questions for COs in humanitarian situations
• Do you have adequate monitoring systems to manage UNICEF humanitarian response
taking into consideration the context, scale and scope of the response?
o Do you know the rough levels of coverage being achieved in key programme activities
with regular frequency? (If No consider prioritising a few key high frequency output
indicators per sector – selected from the Indicator Guide - and including them in
UNICEF cooperation agreements using the PCA Monitoring and Reporting Addendum
tool)
o Do you have some form of systematic verification of the quality of implementation? (If
No then consider using Field Monitors)
o Do you have an overview of the financial, human resources and supply support to
programmes and where the gaps are? (If No then consider using SitRep format within
Humanitarian PM)
o Do you know the status of cluster/sector coordination processes? (If No consider
using Cluster-Sector Coordination Milestone Monitoring Tool)
o Are you collaborating with other partners to get key outcome level data?
• What can UNICEF build upon and link up to; for example, monitoring systems that are in
place or planned by national and international partners, cooperation agreement partners
and wider Cluster partners?
• Who in the CO will be responsible for implementing, supporting and reporting on HPM?
• Are additional resources (human and financial) needed for UNICEF contribution to
Humanitarian PM?
o If yes can CO resources be reallocated
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
9
o Are additional Humanitarian PM resources included in any appeal documents (this
is highly recommended).
o Is HPM surge support needed
• See more under Accountabilities for CO Humanitarian Performance Monitoring
Definitions
Emergency; A situation that threatens the lives and well-being of a population and requires extraordinary action to ensure their survival, care and protection (UNICEF PPPM 2009, and CCCs as per CF/EXD/2010-02, 2010) Complex Emergency: A humanitarian crisis in a country, region or society where there is total or considerable breakdown of authority resulting from internal or external conflict which requires an international response that goes beyond the mandate of any single agency/or the ongoing UN Country Programme (IASC 1994). Levels of Emergency Response3: Level 1: the scale of the emergency is such that a Country Office can respond using its own staff, funding, supplies, and other resources, and the usual RC/HQ support Level 2: the scale of the emergency is such that a Country Office needs additional support from other parts of the organization (HQ, RO, and COs) to respond and that the RO must provide leadership and support. Level 3:the scale of the emergency is such that an organization-wide mobilization is called for
3 Taken from Ex Dir CF/EXD/2011-01: UNICEF’s Corporate Emergency Activation Procedure
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
10
2. Guide to Strengthening Humanitarian Performance Monitoring
Steps to Strengthen CO Humanitarian Performance Monitoring
Q&A Guide on Using Tool
Contents of this section
• Why is this important (what the Country Management Team should know)
• What does it take to do it
1. As preparedness actions;
2. Following a sudden onset emergency4
3. During chronic humanitarian situation
Why is this important (what the Country Management Team should know)
• This section is a quick reference guide to actions to strengthen CO Humanitarian
Performance Monitoring (PM).
• It recognises that there are different contexts for implementing Humanitarian PM and will
consider steps; as preparedness actions5; following a sudden onset emergency, and
during chronic humanitarian situations
What does it take to do it
1. As Preparedness Actions
The following actions should take place both as part of preparedness and immediately
following the decision to implement Humanitarian Performance Monitoring.
Accountabilities (Recommend decision by Deputy Rep)
Decide who within the CO will be:
• The focal point / lead and who will be involved in CO Sudden Onset Response Planning
(where a Response Plan is required) - recommend Dep Rep; Emergency Officer and CO
Planning unit – with someone from the monitoring section
• The focal point / lead for Humanitarian PM (recommend CO Monitoring unit but will
depend on capacity of CO)
• The focal point for CO SitReps (recommend staff with link to CO Monitoring unit;
Emergency Officer and Programme Sections)
4 Following a sudden onset emergency the priority for which steps to take would differ depending on when the
decision to implement Humanitarian PM is taken; i.e. Immediately following the sudden onset trigger; 4 weeks
after the sudden onset trigger; 8 weeks etc. At some point during a chronic emergency. The UNICEF staff leading
for Humanitarian PM needs to adapt the steps and priorities to the CO priorities. 5 Preparedness actions currently outside the EW/A system
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
11
Assessing Existing CO Monitoring mechanisms (Recommend CO Monitoring unit)
Examples of questions:
• Is latest baseline information on population centralised and easily accessible – If not
recommend CO Monitoring unit collates
• Does baseline info provide data against age groups specified within the CCC
commitments6 - If not recommend map gaps and possible solutions for finding data
High Frequency monitoring
• Can current CO monitoring mechanisms provide information on delivery of UNICEF
supplies - If not recommend map gaps and possible solutions
• Can current CO monitoring mechanisms provide information on UNICEF interventions in
terms of numbers of people reached? - - If not recommend map gaps and possible
• Where sector working groups / clusters are used can current CO monitoring mechanisms
provide information on UNICEF progress in meeting sector/cluster lead agency
accountabilities - If not recommend map gaps and possible solutions
• Do current CO monitoring mechanisms provide a structured process for Field Monitoring
visits to UNICEF intervention sites, that includes a qualitative component and
discussions with end-users. I.e. Use and usefulness of supplies; bottlenecks; unintended
negative impacts (exclusions, ‘Do No Harm’) - If not recommend map gaps and possible
solutions
Resourcing (Recommend decision/plan by Deputy Rep in coordination with monitoring
section)
• Based on the gaps and solutions from the Assessment of existing CO monitoring
mechanisms - What extra UNICEF monitoring and reporting human and financial
resources would be needed to;
o Implement higher frequency monitoring in a chronic humanitarian situation
o Implement higher frequency monitoring during a Level 2 or Level 3 sudden onset
emergency response
2. Following a Sudden Onset Emergency
First week
Actions as detailed in Preparedness Actions above
• Accountabilities
• Assessing Existing CO Monitoring mechanisms
• Resourcing
In addition:
Determine UNICEF role in Rapid Needs Assessments (Recommend Deputy Rep in
coordination with CO monitoring unit)
• Accountability for Response coordination and overall information management lies with
OCHA (in coordination with the national government)
6 See Target/Baseline calculator in Indicator Guide
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
12
• At present draft OCHA guidelines7 are seeking to agree the scope of the initial Rapid
Needs Assessment (OCHA accountability) to one or two (depending on size of
emergency) – desk based, qualitative assessment “guestimates” of the response ‘Needs’
to inform the Flash Appeal8 and the revised Flash Appeal.
• Within the proposed MIRNA framework a Needs Assessment survey would be the
responsibility of the country cluster/sector lead agency
• If this context immediate steps are required for the CO to secure the resources (both
human and financial) to carry out a Needs Assessment survey
• Necessary resources should be included in the appeal. Additional dedicated surge staff
would be required to lead the survey(s). An option to consider is ‘contracting out’ the
Needs Assessment survey work, e.g. to CDC (Centre for Disease Control) who can
adapt questions to suit the sector and context.
Agree SitRep Reporting format9 (Recommend decision by Deputy Rep in coordination with
Rep and Regional Office - REA)
• Decision to adopt revised SitRep format
• Task staff member with production of SitRep
• Support SitRep production through clear communication to Programme and Operational
sections, at national, provincial and zonal levels (where applicable)
• SitRep reporting for weeks 1-4 will likely include programme supplies and inputs; and
progress on key operational functions (resource mobilization, supply and HR)
o At week 2-3 SitReps to begin to include progress on UNICEF cluster/sector lead
coordination progress
o After week 4 SitReps can begin to report progress from implementing partner reports
(providing framework for Humanitarian PM has been established)
Week 2-3: Establishing Humanitarian PM framework
Align and strengthen UNICEF cooperation partner monitoring and reporting – Priority
– low cost (Recommend CO Monitoring unit lead, supported by relevant Programme
Sections)
• Prioritize Humanitarian PM (CCC) programme output indicators10 that can realistically be
monitored by implementing partners with the available resources (i.e. 2 or 3 per sector
selected from the Indicator Guide).
• Validate with Programme sections (and ideally with potential UNICEF implementing
partners as time allows. An ideal alternative is to share the UNICEF priority indicators in
advance, noting that these are not the limit of the UNICEF planned interventions)
• Introduce the priority 2-3 indicators into relevant UNICEF cooperation agreements with
partners11, along with agreement that partners will report at a higher frequency (i.e.
monthly following sudden onset; quarterly in chronic emergencies)
7 Multi-Cluster Initial Rapid Needs Assessment guidelines (MIRNA) draft 18April2011
8 For Flash Appeal guidelines see
9 See SitRep format of Humanitarian PM toolkit
10 See Indicator Guide of Humanitarian PM toolkit
11 See PCA Monitoring and Reporting Addendum tool of Humanitarian PM toolkit
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
13
Establish Field Monitoring Visits to UNICEF projects: 2 options – medium cost
(Recommend decision on option by Deputy Rep in coordination with monitoring section)
A) UNICEF Programme staff visit programme sites to verify cooperation partner reports as
well as discussions with end-users. I.e. Use and usefulness of supplies; bottlenecks;
unintended negative impacts (exclusions, ‘Do No Harm’);
B) Where there are access issues or not enough UNICEF capacity for visits – hire Field
Monitors12 (either through an organizational SSA or directly through SSAs with individuals)
to carry out Field Monitor Visits on behalf of UNICEF
Develop a Simplified IMEP and Results Framework – low cost (recommend CO
Monitoring unit)
• Within the first 2 weeks the CO is likely to produce a CO Sudden Onset Response plan13
This is outside the scope of the Humanitarian PM toolkit and the format and guide is
contained in the Emergency Response toolkit
• The Results Framework expands on the Sudden Onset Response plan and includes
both high and low frequency indicators.
Challenges and Solutions Preparedness / First Week
Potential Challenges Possible Solutions Where Challenges not addressed
Accountabilities for
Humanitarian PM not clear
• Early decision (Dep Rep) on
who should lead based on CO
monitoring capacity
• CO Monitoring section says its
Programme section responsibility -
Programme sections don’t have
time/capacity/existing high frequency
monitoring mechanisms.
• Humanitarian PM does not happen
• Poor quality SitRep – adverse effect on
resource mobilization.
Assessing existing CO
monitoring mechanisms
does not happen / takes
too long
• Assess as part of
preparedness and during 1st
week after decision to
implement Humanitarian PM
• Suitable / appropriate elements of
existing CO monitoring not utilized for
Humanitarian PM
Resourcing
• Resourcing (human and
financial) considered in
appeal and in country plan
• Insufficient CO monitoring capacity
• Insufficient funds for Field monitoring
visits (through 3rd
party organization)
• Poor quality SitRep – adverse effect on
resource mobilization.
12
See Field Monitoring Visits - Contracting and Managing Humanitarian PM toolkit 13
See Sudden Onset Response Plan – Emergency Response Toolkit
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
14
Potential Challenges Possible Solutions Where Challenges not addressed
Sit Reps
No-one to write
• Early decision (Dep Rep) on
who should lead for report
writing with sufficient capacity
and links to monitoring,
programmes and Emergency
Officer
• Various staff write SitReps, lack of
continuity in format and content.
• Poor quality SitRep – adverse effect on
resource mobilization.
SitReps
Format does not include
programme performance
tables
• Establish framework for
Humanitarian PM
• Early decision (Dep Rep) to
adopt SitRep version with
programme performance
tables
• CO unable to report beyond inputs and
cluster/sector information where
country cluster/sector lead agency.
• Mixing of UNICEF and cluster/sector
progress.
• Poor quality SitRep – adverse effect on
resource mobilization. External
criticism
Week 2-3
Assess UNICEF progress in country cluster/sector coordination14 where UNICEF
country cluster/sector lead agency
• (Recommend led by CO Monitoring unit)
• After start of sector working groups / cluster: Can be carried out periodically through one-
to-one interviews/discussion with UNICEF cluster/sector coordinator/focal point.
• Where sector working groups / clusters have been operational for some time : Utilise one
of the more in depth cluster / sector coordination tools (e.g. Q-SAC developed by
APSSC)
Supporting Cluster/Sector performance monitoring where UNICEF country
cluster/sector lead agency
• (Recommend led by programme section in coordination with Monitoring unit)
• The PCA Addendum tool of the Humanitarian PM toolkit supports cluster/sector
performance monitoring through inclusion of the commitment to report:
o In line with cluster-sector reporting frequency
o To country Cluster/sector lead agencies
o Using the OCHA 3W.
• Cluster/Sector performance monitoring can be further supported through the sharing of
UNICEF priority programme performance indicators for use as a starting point for
clusters/sectors in agreeing what to monitor.
• CO monitoring sections should support UNICEF cluster/sector coordinators / focal points
to develop and agree realistic, appropriate and useful performance monitoring indicators.
Week 4 onwards
14
See Cluster/Sector Coordination Milestone monitoring tool of Humanitarian PM toolkit
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
15
• CO SitReps to begin to include programme performance progress tables – incorporating
information from UNICEF cooperation partner progress reports (NB. This will only be
possible providing Humanitarian PM has been established – particularly prioritising
indicators and including them in PCAs)
Week 12 or 16 (approximately)
• External CO progress report to include latest information on 2-3 prioritized indicators per
sector with latest assessment results of status of UNICEF meeting cluster lead agency
accountabilities.
At week 26 or 52 (as appropriate)
• Outcome survey of programme results – carried out on inter-agency basis
3. Chronic Humanitarian Situations
• The Humanitarian PM toolkit can be adapted to ‘chronic’ humanitarian situations,
however the variety of chronic situations means a higher level of adaption is required.
• Ongoing work includes adapting HAC planning, mid-year and end of year reporting
requirements to reflect more standardized Humanitarian PM
• A different definition of ‘high frequency’ is likely than in sudden onset responses (i.e.
quarterly or twice yearly)
• Chronic humanitarian performance monitoring should try to build on existing planning
and reporting mechanisms wherever possible.
The following steps outlined above should also be considered for Humanitarian PM in
chronic emergency contexts
• Agree SitRep Reporting format
• Align and strengthen UNICEF cooperation partner monitoring and reporting
• Standardise / Establish regular Field Monitoring Visits to UNICEF projects
• Assess UNICEF country Cluster/sector coordination progress
Challenges and Solutions Chronic humanitarian situations
* Please share Challenges and Solutions through the Humanitarian PM Community of
Practice for inclusion in future drafts of the Humanitarian PM guide.
http://intranet.unicef.org/CoP/EMOPSHPM/CommunityContent.nsf
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
16
Accountabilities for CO Humanitarian Performance Monitoring
Q&A Guide on Using Tool
Contents of this section
• Why is this important (what the Country Management Team should know)
• What does it take to do it
• Establishing Humanitarian PM
• Options to consider when applying
Why is this important (what the Country Management Team should know)
• The clarification of CO accountabilities for establishing and leading for Humanitarian PM
is essential.
• Without clear accountabilities Humanitarian PM will always be ‘someone else’s job’ and it
will not happen.
• The following key decisions are required at the CO Deputy Rep / Senior Management
level
• Tasking of Humanitarian PM within CO, including production of SitRep
• Reallocation of CO resources / including Humanitarian PM into appeals / proposals
• Request for additional Humanitarian PM surge support where necessary
• Endorsement of the revised SitRep format as a CO management tool to inform the
response, document UNICEF CO progress and mobilize resources
• Senior CO management support is essential for Humanitarian PM to have any chance of
successful implementation and providing progress results for UNICEF
• The following are recommendations15 for CO Accountabilities, recognizing that a
decision will based on a CO staff and monitoring capacities
What does it take to do it
Key Decisions and Actions Proposed CO Accountability
Lead Supported by
Senior Management Decisions
Dep Rep
Emergency Officer /
Head of monitoring
15
Recommendations assumes that the CO has sufficient Monitoring capacity to lead for establishing Humanitarian PM – otherwise additional Humanitarian PM capacity / surge support will be required
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
17
Key Decisions and Actions Proposed CO Accountability
Lead Supported by
Tasking of Humanitarian PM within CO
Reallocation of CO resources / including
Humanitarian PM into appeals / proposals
Dep Rep Emergency Officer /
Head of Ops
Endorsement of the revised SitRep format Rep / Dep Rep /
Emergency Officer Emergency Officer
Establishing Humanitarian PM
Tasking of SitRep production
Dep Rep / Emergency
Officer
Emergency Officer16 /
Head of monitoring
Development of Simplified IMEP and Results
Framework from: a) Response Plan; OR b)
Existing Country Plan
Monitoring Unit Programme Sections
Prioritizing 2-3 indicators per sector for high
frequency monitoring (i.e. monthly) Monitoring Unit Programme Sections
Development of PCA Monitoring and Reporting
Addendum Monitoring Unit
Use of PCA Monitoring and Reporting
Addendum Programme Sections Dep Rep
Ensuring UNICEF PCA partners report frequently
against priority indicators Programme Sections Dep Rep / SitRep
Completion of regular Sit Reps Reports Officer17 Dep Rep / All
sections
As required:
Establishing Field Monitoring mechanisms: either
a) 3rd party monitoring organization or direct hire
a) Monitoring Unit
16 Where a CO has an Emergency Unit / Specialist 17
Reports Officer may be part of Communications (external reporting), or the monitoring unit (donor reporting). It is recommended that CO clearly tasks SitRep production to staff with a strong operational link to both the monitoring unit and programme sections.
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
18
Key Decisions and Actions Proposed CO Accountability
Lead Supported by
b) Sector specific monitoring agreements b) Programme Sections Monitoring Unit
As required:
Management / tasking of Field Monitors / 3rd party
monitoring organization
Monitoring Unit Programme Sections
Proposing UNICEF HPM priority programme
indicators to relevant clusters/sector groups
Programme Sections
Monitoring Unit
Completing high frequency assessment of UNICEF
progress in meeting CLA accountabilities
Monitoring Unit
Cluster/Sector
Coordinators / IMs
Inputting / Designing and implementing outcome
survey (through Inter-Agency)
Monitoring Unit
Cluster/Sector
Coordinators
Maintenance of Humanitarian PM Monitoring Unit Dep Rep
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
19
Challenges Solution
Humanitarian PM is ‘someone else’s
job’
• Clear tasking of roles from CO Senior
Management
Additional monitoring work for:
a) CO
b) Monitoring section
c) Programme sections
d) Implementing Partners
a+b) Obtain extra monitoring capacity –
Humanitarian PM surge support
c) Monitoring sections support Programme sections
d) Priority indicators (2-3 / sector) for PCAs + Light
reporting format for PCA partners– share in advance
Existing / regular ole of CO Monitoring
section is not suited for higher
frequency humanitarian monitoring
• Obtain extra monitoring capacity – Humanitarian
PM surge roster through EMOPS
• CO monitoring section buy-in essential
SitRep is weak
• Dep Rep supports SitRep production
• SitRep is tasked to staff with sufficient capacity
• Humanitarian PM is established to produce
monitoring outputs for SitRep
* Please share Challenges and Solutions through the Humanitarian PM Community of
Practice for inclusion in future drafts of the Humanitarian PM guide.
http://intranet.unicef.org/CoP/EMOPSHPM/CommunityContent.nsf
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
20
Simplified Humanitarian Integrated Monitoring and Evaluation Plan
(IMEP) and Results Framework
Q&A Guide on Using Tool
Contents
• Why is this important? (What the Country Management Team should know)
• What the User Should Know
• What does it take to do it?
• Simplified IMEP Template - tool
• Simplified IMEP Diagram
• Results Framework - tool Why is this important (What the Country Management Team should know)? • The Humanitarian IMEP and Results Framework are intended for use following a sudden
onset emergency or significant scaling up of humanitarian response with increase in CO resources requiring increased monitoring and evaluation activities in addition to those in the regular annual IMEP/Results Framework.
• Both the Simplified Humanitarian IMEP and the Results Framework are internal UNICEF planning documents.
• A simplified presentation of the IMEP is useful to explain to donors the measures UNICEF is taking for accountability and performance management and to manage expectations. See Simplified IMEP diagram for example.
• Both the Simplified Humanitarian IMEP and the Results Framework can be a reference for UNICEF advocacy at Inter-Agency level to (1) promote a coordinated cross-sectoral planning of major monitoring and evaluation activities and (2) promote agreement within clusters on 2-3 prioritized indicators for high frequency monitoring.
The Simplified Humanitarian IMEP • The purpose of the Simplified Humanitarian IMEP is to ensure that adequate resources
are allocated to performance monitoring and evaluation of the humanitarian response.18 Note that Appeal formats encourage M&E resources to be integrated in project sheets and not appearing as a separate line.
• The range of Humanitarian PM activities in the IMEP will depend on the country context, including level of resources available and the degree to which major monitoring activities take place at the Inter-Agency level and are therefore coordinated and led by OCHA.
• The CO needs to take strategic decisions on the UNICEF role in Needs Assessment as detailed in Steps to Strengthen CO Humanitarian PM.
• In ongoing humanitarian situations, the Humanitarian PM should be integrated in the annual IMEP.
The Results Framework • The Results Framework is the link between the Inter-Agency Appeal, the CO Sudden
Onset Response Plan and the IMEP. It re-states the UNICEF-supported Expected Results (Outcome level) and maps out the indicators for high and low frequency performance monitoring against expected results.
18 The UNICEF Programme Policy and Procedures Manual advise that the older recommendation for monitoring
and evaluation resource allocation of 2-5% of Country Programme expenditure for activities is too low.
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
21
• Ideally the identification of priority. High frequency indicators will be in the Sudden Onset Response Plan. If this is not the case the Results Framework can clarify high frequency indicators.
• The critical issue for Humanitarian PM is that the Response Plan and/or Results Framework prioritise 2-3 high frequency Indicators per sector that UNICEF and Implementing partners can realistically track.
• These priority high frequency Output Indicators should be included in the UNICEF PCAs Monitoring and Reporting Addendum, SitReps, Humanitarian Progress reports and on to CO Annual Report and Appeal Reports
• The Results Framework also encourages identification of lower frequency Outcome Indicators that can be monitored through survey work. These can be further defined at a later stage in the response as necessary.
What the user should know? The Simplified Humanitarian IMEP • The IMEP Template should be adapted to include only those M&E activities that are
appropriate and feasible to the humanitarian situation.
• Links to additional information on Needs Assessment and Humanitarian Evaluation are
shown in the IMEP Template.
The Results Framework • The scope of the Results Framework, like the Sudden Onset Response plan, is
focused on UNICEF-supported response with cooperation agreement partners (INGOs, NGOs, CSOs, government etc.). Reference to the Cluster/Sector Target is intended to help situate and distinguish overall coverage from UNICEF-supported coverage. The Results Framework does not serve as a framework for monitoring the Cluster/Sector response.
• The Expected Results in the Results Framework are drawn from the UNICEF Response Plan and should align with CCC programme commitments that are relevant to the country context. Where this formulation is weak or missing from the Response Plan, the Results Framework should clarify and ensure that results are measurable and have appropriate indicators that can be realistically monitored at higher frequency (i.e. monthly through implementing partner reporting and field visits) as well as lower frequency (i.e. bi-year or annual through an outcome survey).
• The priority Output Indicators (for high frequency monitoring) are mapped to the Expected Results (not Activities). This means fewer indicators. Potential Challenges Solutions
Too many indicators,
identified without
consideration of
practicalities / capacities /
cost of data collection
• Indicators should be few and remain at a higher
level
• If you have too many indicators it is likely results
have been framed at too lower level - reframe
• Activity indicators are NOT necessary or helpful in
the Results Framework – though it is recognised
that Programmes may need their own work-
planning to the activity level
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
22
• A selection of Output Indicators for high frequency monitoring, mapped onto CCC benchmarks, has been developed as part of the Humanitarian PM Toolkit - Indicator Guide
• Priority indicators should be validated with programme sections however care should be taken that the validation process does not mean an unrealistic increase in numbers of indicators, or a focus on indicators that will not yield information for an extended period.
• Basic information on the Target Affected Population (numbers, location) should be available from the Inter-Agency Needs Assessment, led by OCHA19 or the equivalent National Government Disaster Management authorities.
• The Humanitarian PM Toolkit Sudden Onset Response Plan and Indicator Guide both include a Target Population Calculator to help determine concrete numbers for specific UNICEF and Implementing partner programme interventions. Potential Challenges Solutions
• UNICEF planning
takes place before the
Needs Assessment is
finished
• Needs Assessment
data is too weak to
effectively plan
• It should be accepted within the CO Planning in and
for a sudden onset response will never have the
same level of accuracy, or have “enough”
information as regular country planning processes.
• A limit should be set on CO planning to obtain a
“good enough” plan to move forward with a proviso
to revisit at a later stage (i.e. after 3months)
• UNICEF Targets will need to be revisited
periodically based on projected funding of the
response and any changes to the overall Response
target
• UNICEF Targets are those that the CO and implementing partners can realistically achieve with its resources (assuming appeal is fully funded). o It can be calculated in real numbers or as an estimated % of coverage of the total
Cluster/Sector response. o UNICEF targets are likely to require adjusting periodically as the evolution of the
context and funding projections become more clear or change.
• Means of Verification for high Frequency Output Indicators will include: o Implementing partner (PCA) reporting (See PCA Monitoring and Reporting
Addendum); o Field Monitoring visits; o Cluster/Sector Coordination milestone assessment tool.
• MoVs for Outcome level indicators will typically be measured through a survey, normally at the Inter-Agency level.
What does it take to do it?
• CO capacity will often determine who is most appropriate to complete the Results Framework and this decision should be made by the Dep Rep/Rep/CMT.
19 The Flash Appeal is usually produced prior to the completion of the Needs Assessment based in pre-crisis data (DHS, MICS, SitAns etc. and best guestimates). The revised Appeal is usually issued after the Needs Assessment with updated
numbers for affected populations (average 3 weeks after an emergency is declared)
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
23
• Where COs have adequate planning and monitoring capacity it is recommended that this be tasked to the Planning Section/Unit (or surge support to that unit) to ensure coherence with the existing Country Plan.
Planning Language used
• The flexibility given to COs to determine the level of their PCRs and IRs means a generic Results Framework suitable for every CO cannot be developed.
• Therefore standard OECD-DAC Results Based Management language has been used with standard definitions - see Defining Terms on the next page.
• COs can map their PCRs and IRs accordingly, see following table
• OECD-DAC Language20
Revised Programme Structure Language21
Can be mapped onto Monitored through
Impact Programme Component Result
• UNDAF result • Inter-agency Appeal Result
Outcome Intermediate Result
• Cluster Strategic Result CCC Strategic Result
• CCC Commitment • UNICEF Partner agreement Outcome
Low frequency indicators --typically monitored through surveys at Inter-Agency level
Output • UNICEF Partner agreement outputs
Higher frequency indicators --typically monitored through reporting by cooperation partners and field visits
Defining Terms Impact -- Positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended. (OECD-DAC p.24) Outcome -- The likely or achieved short-term and medium-term effects of an intervention’s outputs (OECD-DAC p.28) Objective -- Intended impact contributing to physical, financial, institutional, social, environmental, or other benefits to a society, community, or group of people via one or more development interventions (OECD-DAC p.20) Output -- The products, capital goods and services which result from a development intervention; may also include changes resulting from the intervention which are relevant to the achievement of outcomes. (OECD-DAC p.28)
Programme Component Result -- (PCR) describes a significant change that is expected, often at outcome level, to which the UNICEF Country Programme will contribute, in partnership with the government, development partners, civil society and others for the sustained realization of development goals and children’s and women’s rights. The PCR should represent a substantive contribution by the UNICEF Programme of Cooperation to priority issues for children (UNICEF Updated Technical Note Sept 2010) Intermediate Results -- describe UNICEF‟s planned contribution, in collaboration with others, to the achievement of the respective Programme Component Results. Intermediate results are normally at
20 UNICEF has traditionally used RBM language – see http://www.oecd.org/dataoecd/29/21/2754804.pdf and is now moving
toward “Revised Programme Structure” 21 “Revised Programme Structure” language, see http://intranet.unicef.org/EXD/Implement360.nsf/0/BF1E629543AA1FB9852578390078AB2E/$FILE/ATTE0D8O.pdf
Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
24
the output level (but may be at outcome level in some country contexts or for some results). The achievement of PCRs is dependent on the achievement of the respective IRs (and in most cases the achievement of results of other partners and on certain assumptions). The UNICEF Country Office, in partnership with others, has a high degree of accountability for their achievement. (UNICEF – Updated Technical Note Sept 2010)
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
25
Simplified Humanitarian IMEP Template (Delete lines and edit as needed)
Key
mile
sto
nes
Activity
Description
(Thematic Scope;
Geographic Coverage)
UNICEF CO Resources
required
(staff, $)
Q1 Q2 Q3 Q4
CAP Process (Flash/Appeal and Mid- and End-year Review)
Sit
ua
tio
n M
on
ito
rin
g
Initial Needs
Assessment
Inter-Agency, All Programme Depends on UNICEF role; consider
surge or secondments. See IA
resources
X
Post-Disaster Needs
Assessment
Inter-Agency, All Programme Important to decide level of
engagement by UNICEF; consider
surge
X X
Rolling needs
assessments
Inter-Agency, All Programme, localized
crisis or new access
CO should participate, allocate staff,
obtain surge as necessary
X X X X X X X X X X X X
Cross-sectoral
Outcome Survey
Inter-Agency, selected Programme
indicators
Significant resources, comparable to
streamlined MICs
X X
Per
form
ance
mo
nit
ori
ng
SitRep (frequency
agreed with RO)
UNICEF, All Programme and
Operations
Reports Specialist for larger
emergencies
XX XX X X X X X
HAC Reporting UNICEF, All Programme Reports Specialist for larger
emergencies
X X
Partner implementation
reporting
UNICEF & implementing partners;
ideally Blusters; All Programme;
Input/Output indicators
IM support as possible X X X X X X X
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
26
Key
mile
sto
nes
Activity
Description
(Thematic Scope;
Geographic Coverage)
UNICEF CO Resources
required
(staff, $)
Q1 Q2 Q3 Q4
CAP Process (Flash/Appeal and Mid- and End-year Review)
Field Monitoring UNICEF & implementing partners;
Ideally Clusters; Programme Quality;
Verification Supplies
IM support as possible; option of pool
of field monitoring staff ORl institutional
agreement for Third-Party monitoring
X X X X X X X X X X X X
Cluster Coordination
Milestones
UNICEF and CC; ideally whole
Clusters involved
Depends on approach; existing M&E
staff or consultant if less frequent
X X X X X X X
Eva
luat
ion
s an
d R
evie
ws
Action Learning
Review
UNICEF; may be with partners; overall
response
Range of options X X X
IA Real-Time-
Evaluation (RTE)
Inter-Agency UNICEF focal point advisable. X X
Other Evaluation Ideally with partners;
Programme-specific
Follows standard evaluation design
considerations.
X X
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
27
Simplified IMEP Diagram
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
28
UNICEF Results Framework – Template with Example
Programme Sector: WASH
Impact: CCC Strategic Result: Girls, boys and women have protected and reliable access to sufficient, safe water and sanitation and hygiene facilities
Outcome Result
From the CO Sudden Onset Response
Plan
Total Affected Cluster/Sector
Target23
Estimated Most
Vulnerable
Population24
(include timeframe)
UNICEF Target25
(include
timeframe)
Key Output Indicator
For High frequency monitoring
(NB. Recommend select 2-3 per
sector for high frequency
monitoring)
Means of Verification
For High Frequency
Monitoring
Key Outcome Indicator.
Low frequency monitoring.
MoV Usually through Inter-
Agency survey after 6 or 12
months
Expected Outcome Result from CO Response
Plan
Maps onto CCC Benchmark
• In the early days of
planning this figure
may not be known
• It is likely to
change as the
situation changes
• Target population
from Total affected
population
• Specific for each
CCC commitment
Expected UNICEF
+ cooperation
partner Result as
% of target
population
Select from Section A of Indicator
Guide (Humanitarian PM Toolkit)
If not done in the Response Plan,
prioritize 2-3 indicators / sector for
immediate inclusion in UNICEF
PCAs and cooperation
agreements
• MoV for Cluster-Sector
Coordination through
‘Milestone’ assessment
tool
• MoV for Programme
Outputs through partner
reporting and field
monitoring visits
• MoV for supplies,
UniTrack
Select from Section B of
Indicator Guide (Humanitarian
PM Toolkit)
Coordination mechanism provides guidance
to all partners on common approaches and
standards; ensures that all critical WASH
gaps and vulnerabilities are identified; and
provides information on who is doing what,
where, when and how, to ensure that all gaps
are addressed without duplication (CCC1)
N/A N/A N/A Achievement of Cluster / Sector
Coordination Milestones
Cluster/Sector
Coordination Milestone
Assessment
Higher quality assessment of
cluster requires a more analytical
evaluative approach; Q-SAC or
similar/ Cluster – Sector
Coordination assessment /
evaluation tool
22
Should refer to data from either National Disaster Management Authorities or OCHA 23
Cluster Target Population, reflect the TOTAL estimated affected population in need by the relevant sub-population group as per the result statement 24
Refer to Baseline / Target calculator in Humanitarian PM Indicator Guide 25
UNICEF + Implementing partner Target Population should be specific to UNICEF + Implementing partners, should refer to the relevant sub-population as per the result statement; (i.e. All affected, OR children 6-59 months; OR new mothers); should be based on an assessment of the capacity and funding of UNICEF and its partners to respond
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
29
Outcome Result
From the CO Sudden Onset Response
Plan
Total Affected Cluster/Sector
Target23
Estimated Most
Vulnerable
Population24
(include timeframe)
UNICEF Target25
(include
timeframe)
Key Output Indicator
For High frequency monitoring
(NB. Recommend select 2-3 per
sector for high frequency
monitoring)
Means of Verification
For High Frequency
Monitoring
Key Outcome Indicator.
Low frequency monitoring.
MoV Usually through Inter-
Agency survey after 6 or 12
months
Children and women have access to at least
7.5 - 15 litres of clean water per day (CCC 2)
1,000,000 people 1,000,000 people 500,000 people # and/or % of affected population
provided with sufficient water of
adequate quality
Cooperation Partner
reporting
Field Monitoring Visits
# and/or % of affected
population provided with
sufficient water of adequate
quality
Hygiene education / information is provided to
70% of women and child-caregivers
pertaining to safe and hygienic child care and
feeding practices (CCC 4)
1,000,000 people 650,000 women and
child caregivers
400,000 women
and child
caregivers
# information 'packages' or #
events reaching # population
to give % total of expected
coverage
Cooperation Partner
reporting
Field Monitoring Visits
% people able to identify 3 key
hygiene practices
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
30
3. Humanitarian Performance Monitoring Tools
Situation Report
Q&A Guide on Using Tool
Contents
• Why is this important? (What the Country Management Team should know)
• What the User Should Know
• What does it take to do it?
• Options to consider when applying
• SitRep template - tool
Why is this important (what the Country Management Team should know)
• Under the revised CCCs, as part of the accountabilities for humanitarian performance
monitoring, all COs in ongoing humanitarian situations26 are expected to produce regular,
standardized SitReps.
• The SitRep template has been re-framed as a management tool comparable to the CO Management Report and useful to COs, RO and HQ in tracking an overview of the response.
• It covers; o The situation analysis in terms of the regional and global policy/political context; o Both programme implementation and operations requirements focusing on few priority
indicators; o Status of cluster coordination and performance data for clusters where UNICEF is lead
• The SitRep template is also designed to streamline and ensure quality, effective and
timely flow of information in emergencies. It is divided into two parts:
o the first for External Use to ensure consistent regular communications with the media,
NatComs, Donor Governments and partners and support resource mobilization;
o the second part for Internal Distribution with more sensitive information and for
internal management, identifying key gaps to be addressed at the CO level including
through RO/HQ support; informing advocacy initiatives undertaken at regional and
global level.
Humanitarian Performance Monitoring activities are intended to feed into and
populate the SitRep. Without some focused humanitarian PM activities the CO SitRep
will be a weak product in terms of programme results
• CO Senior Management is responsible to clear the SitRep and ensure quality control before
sending it to OSPCEN for dissemination to internal and external partners • The SitReps provide an important record on the progress of the response and thus basis for
any major donor report (e.g. a 6-month report in a major emergency) or evaluation.
26
The simplest way to determine this is to consider access to humanitarian funding through Flash Appeal, Consolidated Appeal Process or Humanitarian Action for Children (HAC – formerly HAR)
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
31
Options to consider when applying
• Frequency of SitReps will vary depending on the phase and type of the emergency and follow a schedule agreed upon by the CO in consultation with the RO and EMOPS
• Daily or Weekly Situation Reports27 are suitable for those COs facing new emergencies or a significant upsurge of humanitarian needs – often accompanied by submission of an Immediate Needs Document (IND), CERF, EPF, Flash Appeal documents. Once the situation has stabilized this will become a monthly SitRep.
• The Daily/Weekly SitRep is meant to be lighter (concise bullet points only) for more frequent updates. Provisions are also made for more limited reporting during the first 1-4 weeks of a new emergency while monitoring systems may not yet be generating programme data.
• Where a CO is responding to multiple emergencies, a decision is required on whether to try to aggregate reporting on the overall humanitarian response or keep them separate. Since planning, funding and CO budget mechanisms are likely to be separate for multiple responses – SitReps may be more useful and clear if they are kept separate.
What does it take to do it
• Decision to establish the schedule for SitRep reporting in consultation with RO and HQ – see Accountabilities for CO Humanitarian Performance Monitoring and Steps to Strengthen CO Humanitarian PM;
• Agree on the priority high frequency programme indicators to be included in the SitRep.
• Until implementing partner reports begin to arrive SitReps should report inputs – see Indicator Guide, Column B of appropriate Section.
• Info from implementing partner reports should arrive after one month signing the PCA
• For countries in chronic crises, reporting should be aligned to CAP/HAR targets.
• Frequency in chronic crises should be agreed between CO, RO, HQ – likely to be quarterly.
• Ensure that the SitRep is used as a common reference for information uses with external media communications and other advocacy efforts ongoing.
• Additional guidance for completing the SitRep are contained within the body of the SitRep template in blue text.
Potential Challenges Solutions
Attribution • UNICEF must clearly acknowledge that it contributes to programme results achieved together with partners
• Programme Section PCA partners should be includes in the relevant tables
Double counting • Every effort and care should be made not to double count across UNICEF sections.
• In instances where larger international NGOs are supporting smaller NGOs (often national), care must be taken to ensure there is not overlap if they are all reporting.
Mixing UNICEF and
cluster/sector
performance information
• A clear distinction should be made between UNICEF and PCA partners results and wider cluster/sector results
* Please share Challenges and Solutions through the Humanitarian PM Community of
Practice for inclusion in future drafts of the Humanitarian PM guide.
http://intranet.unicef.org/CoP/EMOPSHPM/CommunityContent.nsf
27
It is recognized that for daily or weekly SitReps (in the intitial period after a sudden onset emregcny response, that the SitRep will focus on the information available such as progress against process for setting up the response, UNICEF inputs (as outlines in the Indicator Guide, section B)
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
32
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
Situation Report (Country) Date: xx
Reporting period: (indicate if daily / weekly / monthly)
Guidance Notes for the Template in Blue below. Further Questions and Answers Guidance at the end of this document Please delete blue guidance instructions and Q&A when completing Overview
• All COs accessing humanitarian funding through HAC are now required to produce regular, standardized Situation Reports (SitReps)
• SitReps should be submitted on a monthly basis; unless otherwise agreed between CO, RO and HQ depending on the humanitarian situation
• More frequent SitReps use a daily/weekly format, less frequent use a monthly format
• The SitRep templates are split into two parts: 1. External: for sharing with a wide audience (partners, NatComs, government donors, etc); 2. Internal (UNICEF only)
• Where there is no change in information provided (i.e. staffing levels, funding levels), the previous report figures can be repeated noting date of last change (“information as of …date…”)
• Tables and sections highlighted in YELLOW may be omitted until information is available
• CO SitReps cannot be prepared in isolation and represent the CO overview of performance management data that captures both the humanitarian situation and UNICEF humanitarian action.
• Without addressing Humanitarian Performance Monitoring a CO is unlikely to be able to produce a good quality SitRep
Sit Rep Support
• The production of the SitRep is supported by the UNICEF Humanitarian Performance Monitoring toolkit and guide – see http://intranet.unicef.org/emops/emopssite.nsf/root/PageCCCPM1
• Ongoing feedback on use and quality of Sit Reps is available from Regional Offices or EMOPS.
For External Use Situation Overview & Humanitarian Needs (sources included to the extent possible)
• Brief contextual analysis – including status of emergency threats (conflict, disaster); political and other developments on the ground
• Key humanitarian issues and key changes since the last SitRep, specifically impact of the emergency on children and women with data disaggregated by sex, age, geographical area wherever possible (estimates for disaggregated data can be given using standard sex/age breakdowns)
• Note any assessments done by UNICEF and/or by Inter-Agency teams, their coverage, focus and key results, as well as major assessments in preparation/ongoing
Inter-Agency Collaboration and Key Partnerships • Humanitarian Coordination Structure -- Brief description of coordination structure in country; this
can stand as a standard description repeated in future SitReps for context; UNICEF engagement in humanitarian coordination structures; key areas/strategies of inter-cluster and inter-agency collaboration and progress in these.
Highlights (for weekly reports only - bullet points with key issues only):
• Changes in the situation (since last SitRep)
• Number of affected population with a focus on children and women
• Key humanitarian needs and challenges
• Major programme results/significant achievements for the reporting period (as estimated percentage
of total affected population)
• Any major programmatic gaps (as estimated percentage of total)
• Any major funding issues
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
33
• Partnerships with INGOs, NGOs – Provide an overview of numbers of partnerships under cooperation agreements by sector reflecting both international and national NGOs.
Programme response • The CO has two presentation phases for reporting programme results; Phase 1) Descriptive text
(narrative), and Phase 2) Results tables including of 2-3 priority indicators per sector
• COs may also opt for a mix where progress is different across sectors.
• COs using descriptive text should have a plan in place to scale up (in capacity and partnerships) in a finite time to be able to use the Results tables.
Phase 1: Descriptive text Used alone when there is little available in terms of monitoring / reporting on the UNICEF programme response, i.e. following sudden onset emergency. From an external communications perspective (i.e. to NatComs and donors) it is important to present information in terms of “estimated population that can be reached” wherever possible. For each programme CCC sector include:
• Planned activities as relevant to the specific commitments under the programme CCC sector
• Any significant process milestones achieved
• Any salient supplies in pipeline or distributed that can be presented in terms of target number of people to be reached, or as soon as there are rough figures on the affected population, in terms of estimated % of the target population. (Note that total value of supplies in pipeline/distributed by sector are presented in a table below).
• Key qualitative aspects from field monitoring visits, i.e. programme approach (gender in humanitarian situation, Communication for Development etc.)
Nutrition Health
WASH
Child Protection
Education
HIV/AIDS Phase 2: Programme results tables and descriptive text
• The results tables should be introduced during chronic emergencies, or latest 4-6 weeks after sudden onset trigger, populated as programme implementation data from partners becomes available.
• The indicators presented below are the recommended priority indicators for monthly monitoring per sector, acknowledging that this depends on programming contexts.
• Experience shows that it is only manageable to track two-three priority indicators per sector, with Child Protection sometime requiring more depending on key issues in the context.
• For guidance on suitable Indicators see Humanitarian PM Toolkit http://www.intranet.unicef.org/emops/emopssite.nsf/root/PageCCCPM1
• Reporting on these same priority indicators throughout the response will feed into any major external CO progress reports.
• Programme Results Table can be used even if targets are still “to be determined” or need to be noted as “preliminary targets”.
• Where UNICEF develops performance indicators in advance of the relevant cluster, UNICEF as cluster member can propose the use of UNICEF determined indicators as a starting point for cluster indicator discussions to facilitate coherence.
• As much as possible data collection should disaggregate data between men, women, boys and girls, though this will more often be possible in ongoing humanitarian situations and/or when population-based surveys are possible.
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
34
Estimated #/% coverage
UNICEF & operational partners Sector / Cluster
UNICEF Target
Cumulative results
Change since last
report
Cluster Target
Cumulative results
Change since last
report
NUTRITION
Children <528 with Severe Acute Malnutrition in Therapeutic Feeding programmes
# 000000
000 ▲▼ # 000000
000 ▲▼ % target 00 % target 00
Children <5 with Moderate Acute Malnutrition in supplementary feeding programmes
# 000000
000 ▲▼ # 000000
000 ▲▼ % target 00 % target 00
Children with SAM under treatment recovered
# 000000 000 ▲▼
# 000000 000 ▲▼
% target 00 % target 00
Comments: Include here any qualitative information and explanation of downward trends as well as any major process milestones including towards capacity development with national partners UNICEF Operational Partners:
HEALTH
Children <5 receiving measles vaccination
# 000000 000 ▲▼
% target 00
Children 6-15years receiving measles vaccination
# 000000 000 ▲▼
% target 00 Emergency affected families receiving 2 ITNs
% target 00
Comments: Include here any qualitative information and explanation of downward trends as well as any major process milestones including towards capacity development with national partners UNICEF Operational Partners:
WASH
Emergency affected population provided with access to an improved water source
# 000000
000 ▲▼
# 000000
000 ▲▼ % target 00 % target 00
% target 00 % target 00 Emergency affected population living in faeces free environment
# 000000 000 ▲▼
# 000000 000 ▲▼
% target 00 % target 00
Comments: Include here any qualitative information and explanation of downward trends as well as any major process milestones including towards capacity development with national partners UNICEF Operational Partners:
CHILD PROTECTION
Children reached through safe environments
# 000000 000 ▲▼
# 000000 000 ▲▼
% target 00 % target 00
Children unaccompanied/ separated from primary care reunified or placed in alternative care arrangements
# 000000
000 ▲▼ # 000000
000 ▲▼ % target 00 % target 00
Emergency affected population reached by efforts to mobilize communities to prevent and address violence, abuse and exploitation, including GBV
# 000000
000 ▲▼
# 000000
000 ▲▼ % target 00 % target 00
Children enrolled in psycho-social activities
# 000000 000 ▲▼
# 000000 000 ▲▼
% target 00 % target 00 Emergency affected population reached by mine/UXO risk education activities
# 000000 000 ▲▼
# 000000 000 ▲▼
% target 00 % target 00
Comments: Include here any qualitative information and explanation of downward trends as well as any major process milestones including towards capacity development with national partners UNICEF Operational Partners:
28
Children 6 to 59 months
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
35
Estimated #/% coverage
UNICEF & operational partners Sector / Cluster
UNICEF Target
Cumulative results
Change since last
report
Cluster Target
Cumulative results
Change since last
report
EDUCATION
School- aged children / adolescents reached by re-established school and/or non-formal learning programmes
1,000,000 # 000000
000 ▲▼ # 000000
000 ▲▼ % target 00 % target 00
School- aged children / adolescents in learning programmes in temporary facilities
1,000,000 # 000000
000 ▲▼ # 000000
000 ▲▼ % target 00 % target 00
School-aged children/ adolescents accessing psychosocial support thru schools
# 000000
000 ▲▼ # 000000
000 ▲▼ % target 00 % target 00
Comments: Include here any qualitative information and explanation of downward trends as well as any major process milestones including towards capacity development with national partners UNICEF Operational Partners:
HIV/AIDS
Emergency affected population with access to appropriate post-rape prevention, care and treatment including use of PEP
# 000000
000 ▲▼
% target 00
Pregnant women with access to prevention, care and treatment including PMTCT
# 000000
000 ▲▼
% target 00
Comments: Include here any qualitative information and explanation of downward trends as well as any major process milestones including towards capacity development with national partners UNICEF Operational Partners:
Where relevant, UNICEF, as cluster lead agency, is responsible for information management of the cluster and sharing overall results achieved by the cluster collectively.
Cluster Coordination • Summarize highlights from more detailed internal section on cluster coordination below including
key milestones, key results in terms of coverage, coverage gaps and challenges. When reporting results for the clusters, it is important to communicate to external audiences that “UNICEF, as cluster lead agency, is responsible for information management of the cluster and sharing overall results achieved by the cluster collectively.”
• In first daily/weekly SitReps after a rapid onset emergency or in crisis phase, if performance monitoring systems are not in place to report cluster level results in terms of affected population reached (i.e. if the option of Results Table 1 is selected), include here key highlights for each cluster on numbers of partners engaged and any estimates on planned clusters targets/coverage and projected gaps.
Supply and Logistics • Figures on total supplies by sector, in pipeline
29 and delivered to partners (USD); this can be a
summary from more detailed internal table on supplies below. • Summary as relevant of In-Kind Donations (USD Value) by programme committed and received
and indicating major partners. Not that an additional row may be necessary • Include narrative on any supply and logistics system arrangements and challenges.
Human Resources • Total human resources mobilized including total number of staff and surge on the ground (noting
stand-by partner organizations)
• Total numbers of staff (temporary and fixed) and surge under recruitment; and total staffing needs unfunded
29
“Pipeline” defined as PGM raised, not yet in UNICEF Warehouse
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
36
Media and communication • Key media messages
• Key media activities/events undertaken and planned
Funding30 • Update on any appeals issued (date, total amount) or in preparation
• Brief statement or summary in table on UNICEF’s funding requirements, funds received to date and remaining funding gap. This can be summarized in text when little information is available and replaced by the tables below when possible.
Appeal Sector
Requirements Funds received Funding gap
Total Cluster
Coordination Total Cluster
Coordination Total Cluster
Coordination
NUTRITION
HEALTH
WASH
EDUCATION
CHILD PROTECTION
GBV
SHELTER & NON-FOOD ITEMS
OTHER
TOTAL
• COs may remove the Cluster Coordination column where the Appeal does not disaggregate cluster costs.
Next SitRep: • Date of next sitrep
For further information, please contact Name Representative Country Office Country Telephone: Facsimile: E-mail:
Name Deputy Representative Country Office Country Telephone: Facsimile: E-mail:
Name Communication Officer Country Office Country Telephone: Facsimile: E-mail:
30
Data for the funds received are available on the PARMO intranet
http://www.intranet.unicef.org/PFO/PFOInfoV3.nsf/Site%20Pages/Page020103) and can be used by the COs to
obtain the total contribution amounts for the contributions that have been received. The COs should report on the
total contribution received, and not just on the programmable amount. The CO can contact PARMO if they have
any queries regarding the contribution amounts.
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
37
Internal distribution only
Issues for Inter-Agency/Governmental Advocacy • Identification of key issues (political and interagency) that impact on UNICEF response and/or
children and women requiring advocacy at HQ/regional levels. Consider IASC, ECHA, Policy Committee, Interagency discussions, as forums HQ can engage with to help advocate on behalf of the CO as well as processes such as PDNA/RF, RTE’s, and changes in humanitarian leadership etc. that need HQ/RO support.
Programme Response • Highlight any critical programmatic issues which are not suitable for an external audience • Key qualitative aspects from field monitoring visits, i.e. unintended impacts, bottlenecks in
implementation etc. • HQ or RO support needed
Cluster Coordination • Adjust table to clusters where UNICEF is Cluster Lead in country.
• Fill in only that which is relevant; it is expected that full achievement of cluster coordination milestones will not be achieved in the first 4-6 weeks.
• It is important in managing communications around cluster performance that these milestones be carefully represented distinguishing that which is specifically UNICEF accountability. This is an issue for CO senior management attention.
• Amend table as necessary (additional columns, rows etc.) where clusters are activated at the sub-national level
Cluster Coordination Progress Milestones
NUTRITION WASH EDUCATION CHILD PROTECTION
GBV
CC function filled [1] Indicate
Yes – dedicated CC Yes – dual role No
(Where Co-lead). Have areas of responsibility been defined and agreed Yes/No
Coord mechanism/ TOR established Yes/No
Cluster Strategy developed Yes/No
Standards promoted Yes/No IM system established [2] Yes/No
Cluster Performance management system agreed Yes/No
Have gaps in response been identified [3] Yes/No
[1] Indicate – Level CC; date arrived/existing staff assigned; duration assignment; or duration of gap in weeks [2] Indicate: Accountability for cluster IM established within cluster staff or CLA office; IM role staffed; 3W maintained and circulated at agreed frequency; Information on coverage collected and shared [3] Indicate: gaps monitored regularly; resources redirected to respond; advocacy undertaken • Narrative: Include any concerns / constraints by sector
• Include Action taken to fill identified gaps in response • HQ or RO support needed
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
38
Supply & Logistics At the onset of an emergency, in first weeks, the following are relevant milestone indicators to report
on:
• First Response Supply Plan completed. See Supply Calculator
• Supply and Logistics Strategy completed. Guide and tool forthcoming, see SD emergency page
• Tracking system for logistics and procurement established.
•
• Table of total value of Supplies in USD by programme CCC sector. • Where relevant an additional row can be added on general support supplies such as office set up
or re-establishment costs, security and telecommunications equipment. • Additional data on Volume and Weight by commodity can be provided in the “Field Supply
Monitoring Table”, if necessary, which can be attached for internal use. • Totals should include both local and offshore procurement as relevant. • This table is exclusive of the value of freight charges.
• Any additional issues or areas of concern not suitable for external audiences
• HQ or RO support needed
Media and Communication • Sensitive issues to be flagged
• Key messages, including for public advocacy
• HQ or RO support needed
Security and Access • Update on security situation and incidents
• Overview of impact on programme implementation and CO operations
• Update on key measures regarding staff safety and security HQ or RO support needed
• HQ or RO support needed
Sector Total Amount
Committed Total Amount Raised PGMs
In pipeline Total Amount
Received In UNICEF
Warehouse�
Delivered to partners -- Cumulative
Delivered to partners -- Since last
report
NUTRITION
HEALTH
WASH
CHILD PROTECTION
EDUCATION
HIV/AIDS
�current inventory = previous including prepositioned + new
“Pipeline” defined as PGM raised, not yet in UNICEF Warehouse
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
39
Human Resources
Surge Capacity
International (TA/FT/Mission/SSA/ERT/RRT)
Total positions In position Identified and arrival awaited
Vacant/Sourcing
National (New TA positions, including clusters
Total positions In position Under recruitment/Hold
# National Staff redeployed in country
Resource Mobilization • Update on any UNICEF or Inter-Agency efforts on fundraising as relevant, donor meetings that
have taken place and planned for. .
• HQ or RO support needed
Finance and Administration • HQ or RO support needed
Information and Communication Technology • HQ or RO support needed
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
40
Indicator Guide (NB. Full Indicator guide is an Xcel document)
Contents of this section
• Why is this important (what the Country Management Team should know)
• What users should know
o Uses for the Indicator Guide
o Structure of Indicator Guide
• Additional considerations in using the Guide
• What does it take to do it / Indicator Guide Walk through
Why is this important (what the Country Management Team should know)
• For a manageable (i.e. realistic and achievable) Humanitarian PM system it is essential
to agree on a prioritized set of indicators that can be monitored at a high enough
frequency to inform the response.
• The Indicator Guide provides a reference for selecting priority high frequency
indicators, for both UNICEF implementing partners and for reporting in the UNICEF
SitRep.
• Experience suggests two or three priority high frequency indicators per sector are
realistic and feasible for partner data collection (with the exception of Child Protection
where more may be required depending on the context).
• Inclusion of two or three priority high frequency indicators per sector have been
incorporated into the new Sudden Onset Response Plan template and guide that has
been developed for the Simplified Standard Operating Procedures (SSOP) on Planning,
Monitoring and Evaluation for Level 3 emergencies.
• As country cluster/sector lead agency, UNICEF should advocate for a similar
agreement on a few priority high frequency performance monitoring indicators with
national government partners and within the cluster/sector
• As cluster/sector member UNICEF should strive to align the monitoring framework of
PCAs with indicators established nationally and /or at the cluster/sector level.
What users should know
• The Indicator Guide is a separate Xcel document with high frequency, input and low
frequency indicators for each CCC programme commitments and benchmarks.
• The Humanitarian PM Indicator Guide was developed from the revised UNICEF Core
Commitments for Children in Humanitarian Action and for advocacy with implementing
partners and cluster/sector members.
• The Indicators Guide is aligned to the 2011 revision of the SPHERE Minimum Standards
in Humanitarian Response and INEE (Inter-Agency Network for Education in
Emergencies) Minimum Standards for Education, both updated in 2011. It is also
aligned with draft IASC Needs Assessment Task Force indicators.
• Selection of priority indicators must of adjusted to fit the country-specific humanitarian
response however some indicators can be standardized globally and these should be
used wherever applicable.
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
41
Uses for the Indicator Guide
Selecting 2-3 priority indicators per sector for high frequency monitoring
• Included in the Sudden Onset Response plan
• Included in relevant PCA agreements through PCA Monitoring and Reporting Addendum
• Included in the SitRep
Developing the Simplified Humanitarian IMEP and Results Framework
Structure of Indicator Guide
• The Humanitarian PM Indicator Guide is structured following the CCC programme
sectors; commitments and corresponding benchmarks
• For each benchmark, the Indicator Guide provides:
o High frequency indicators (Part A) -- These are intended for lower cost data
collection through partner reporting.
o Proposed indicators for high frequency monitoring are shaded in yellow. NB.
Child Protection has more indicators.
o They will provide lower levels of precision in measuring results and should be
accompanied by systematic Field Monitoring.
o It is recommended that UNICEF advocate for the national and cluster/sector
agreement on prioritization of the same priority high frequency indicators for
monitoring by cluster/sector level members.
o Initial process and input indicators (Part B) -- These are suitable for inclusion in
SitReps until partner reporting data begins – typically one month after PCAs are
signed.
o Outcome indicators (Part C) – These are normally population-based surveys for
data collection and will therefore be lower frequency and higher cost.
o In most cases, data collection on outcome indicators should be cross-sectoral /cross-
cluster for greatest efficiency and should figure in Inter-Agency monitoring plans
• Guidance is provided on disaggregation for each indicator, Means of Verification or
source, how indicators are calculated and any considerations specific to monitoring the
programmatic activity in question.
• Cluster/Sector Coordination indicators are standardized across all sectors where
UNICEF is country Cluster/Sector Lead Agency
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
42
Additional considerations in using the Guide
• Outcome indicators in the Guide are intended for measurement through a representative
population-based survey (as opposed to service-based implementation monitoring).
• Such surveys may be limited by security-related access issues. Where such situations
prevail and it is impossible to carry out a representative survey of the full affected
population, it is possible and meaningful to carry out a survey in accessible areas as long
as the limitations and population not covered are clearly reported.
• Where CO capacity allows it is recommended that surveys are led by the CO Monitoring
unit and validated through discussion with the relevant programme sections
• This process requires both the lens of programme priorities and the lens of monitoring
methodologies and what is feasible in the context (i.e. programme staff to clarify
priorities; monitoring staff to remain realistic in terms of how much data and how many
indicators can be monitored on a monthly basis).
Potential Challenges Solutions
Prioritizing indicators (i.e. a tendency to be over ambitious)
Clear messaging within UNICEF on UNICEF and implementing partners capacity to monitor
Piloting shows 2-3 indicators / section is a best case scenario
Focus on what information is “Need to Know”, over what is “Nice to Know”
Consistencies on integrated programming across sectors
Cross reference priority indicators between sections (i.e. WASH in Schools) and agree with sections Who will monitor and report
It is important to define how organizations will interpret the "affected population" and what monitoring will cover the "affected population" or "affected areas" – for UNICEF, UNICEF implementing partners and ideally as Clusters/Sectors
Efficiencies in cross-sectoral/cross-cluster monitoring
It is often challenging to coordinate monitoring activities across sectors/clusters. It is advisable to focus on the greatest efficiencies, i.e. on high cost monitoring activities such as cluster/sector surveys and larger third-party monitoring systems
Monitoring C4D activities are more efficiently monitored as a cross-sectoral activity. They are rarely prioritized for high frequency monitoring which is difficult; however they can be effectively addressed through survey methodologies and Field Monitoring.
* Please share Challenges and Solutions through the Humanitarian PM Community of
Practice for inclusion in future drafts of the Humanitarian PM guide.
http://intranet.unicef.org/CoP/EMOPSHPM/CommunityContent.nsf
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
43
What does it take to do it / Indicator Guide Walk through
Structure of Indicator Guide
• The Humanitarian PM Indicator Guide is structured following the CCC programme
sectors; commitments and corresponding benchmarks
• Each CCC commitment (column A) is shown with the CCC benchmark (column B) – the
benchmark is broken down where it addresses more than a single standard
• Column C indicates whether the indicator is considered a Priority (for high frequency)
with a guide for adaption to the country context
• Column D is only completed where relevant and shows simplified language for
representing the indicator in the SitRep
Columns A - D
A B C D
Reference in CCCs
CCC Commitment CCC Benchmark Prioritization and Adaptation of
Indicators to Country Context
Simplified presentation of Indicator
in SitRep
EDUCATION
E1. Effective leadership is established for
education cluster/ inter-agency
coordination (with co-lead agency), with
links to other cluster/sector coordination
mechanisms on critical intersectoral issues.
Coordination mechanism provides
guidance to all partners on common
standards, strategies and approaches,
ensuring that all critical education gaps and
vulnerabilities are identified, and provides
information on roles, responsibilities and
accountability to address all gaps without
duplication.
- Priority indicator for monthly reporting
where UNICEF is CLA
- Adaptation in additional milestones but
core milestones will be standardized
See Cluster/Sector Coordination
Milestone Monitoring Tool --
Humanitarian PM Toolkit [hyperlink]
- SitRep includes text description of
coordination milestones in external section;
table with milestones in internal section
E2. Children including preschool age
children, girls, and other excluded
children, access quality education
opportunities
Schools re-opened and child-friendly and
adolescent-friendly emergency non-formal
programmes, including play and early
learning for young children, established for
affected communities
- Priority indicator for monthly reporting
- Standardized; note indicator is framed
broadly enough to capture different
approaches
# and % of school-aged children in
affected areas in schools/learning
programmes
Note includes aggregate of schools (this
row) and non-formal education (next rown
down)
Core Commitments for Children in
Humanitarian Action
Education
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
44
A) Columns E-H – High Frequency Monitoring Indicators
• Columns E represent high Frequency Monitoring indicators – these are lower quality and higher frequency indicators – typically monitored monthly (following sudden onset, quarterly in chronic) and data collected through Routine Project monitoring (i.e. UNICEF PCA partner reporting).
• High Frequency Monitoring indicators in Column E can be selected for: o Sudden Onset Response Plan (2-3 per sector, those recommended shown in yellow) o Included in the relevant PCAs – using the PCA Monitoring and Reporting Addendum
(2-3 per sector) o Included in the SitRep (2-3 per sector) o Included in the Simplified Humanitarian IMEP and Results Framework o Denominators for High Frequency Monitoring indicators will be the relevant
population groups of the affected population within UNICEF and PCA partner project areas.
o Where relevant, and for reference, the indicator is mapped onto the corresponding Needs Assessment Task Force (NATF) indicators (v. May 2011).
• Column F shows indicator disaggregation
• Column G shows the data source
• Column H provides any relevant guidance notes for using the indicator Columns E-H
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
45
B) Column I – Input Indicator
• Columns I shows Input indicators
• Data source will likely be the CO Supply / Logistics unit and can be extracted from UniTrack where established (after 2012 they will be available from VISION).
• They are suitable for inclusion in SitReps until partner reporting data begins – typically one month after PCAs are signed following a sudden onset response (assuming Humanitarian PM is/has been established).
Column I
A B I
BFor monitoring and reporting until
PCA partner progress reports
received
Reference in CCCs Lowest quality indicators
CCC Commitment CCC Benchmark Input indicators and process milestones of relevance for
initial weeks of response in rapid onset
EDUCATION
E1. Effective leadership is established for
education cluster/ inter-agency
coordination (with co-lead agency), with
links to other cluster/sector coordination
mechanisms on critical intersectoral issues.
Coordination mechanism provides
guidance to all partners on common
standards, strategies and approaches,
ensuring that all critical education gaps and
vulnerabilities are identified, and provides
information on roles, responsibilities and
accountability to address all gaps without
duplication.
----
E2. Children including preschool age
children, girls, and other excluded
children, access quality education
opportunities
Schools re-opened and child-friendly and
adolescent-friendly emergency non-formal
programmes, including play and early
learning for young children, established for
affected communities
- school supplies in pipeline or delivered for # of children
- # of tents provided for temporary facilities with capacity for # of
children and adolescents
Core Commitments for Children in
Humanitarian Action
Education
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
46
C) Columns J-M – Low Frequency Outcome Indicators
• Columns J represent Outcome indicators - normally population-based surveys and will
therefore be lower frequency and higher cost.
• In most cases, data collection on outcome indicators should be cross-sectoral /cross-
cluster for greatest efficiency and should figure in Inter-Agency monitoring plans
• Low Frequency Monitoring indicators in Column J can be selected for: o The Simplified Humanitarian IMEP and Results Framework o Population based surveys o Where relevant, and for reference, the indicators are mapped onto existing Multi-
Cluster Indicator Survey indicators o Where relevant, and for reference, the indicator is mapped onto the corresponding
Needs Assessment Task Force (NATF) indicators (v. May 2011).
• Column K shows indicator disaggregation
• Column L shows the data source
• Column M provides any relevant guidance notes for using the indicator
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
47
PCA Monitoring and Reporting Addendum
Q&A Guide on Using Tool
Contents of this section
• Why is this important (what the Country Management Team should know)
• What users should know
• What does it take to do it
• PCA Monitoring and Reporting Addendum Templates – tool (NB. Templates are pre-
completed with recommended priority high frequency indicators. CO should change as
necessary – refer to Indicator Guide)
Why is this important (what the Country Management Team should know)
• The 2010 version of the PCA legal document specifies a default reporting requirement of
3months for financial and 6 months for the narrative.
• This can, and should, be adapted for higher frequency monitoring during humanitarian
situations through selection of a few priority high frequency indicators per section and
including them in PCAs, along with a ‘lighter’ higher frequency31 reporting requirement
• The PCA Addendum agrees a systematic approach to monitoring and reporting by partners, at an adequate frequency and according to a minimum set of indicators.
• This tools supports implementing partners by having fewer indicators – recognising the
realities of data collection, partner (and UNICEF) monitoring capacity and use of
collected data in Humanitarian Situations
• Incorporating standard priority indicators into the PCA should support the CO
Programme Review Committee approval process. I.e. Ensuring that monitoring
frameworks are simplified and clear therefore less likely to become a reason for delay.
• The Indicator Guide proposes priority indicators for inclusion in the Sudden Onset
Response Plan, PCAs (using this tool) and SitReps
What users should know
• The PCA Monitoring and Reporting Addendum will be primarily used by UNICEF
programme Sections (it is recommended that they are prepared by CO monitoring
sections)
• It provides a platform for discussion and agreement on a more frequent and lighter
reporting process for UNICEF PCA partners based on a number of key results indicators
that aim to be monitored throughout the UNICEF response.
• It is an easy reference tool to support programme sections and implementing partners in developing a monitoring framework
• It ensures UNICEF PCA monitoring remains realistic by identifying a few priority sector
indicators that are standardized across partners.
• The PCA Addendum is made up of,
31
UNICEF and Implementing Partners should agree an appropriate frequency for reporting. Following Sudden onset emergencies this may be monthly. During chronic emergencies this may be quarterly. It should be aligned with cluster/sector reporting requirements where possible.
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
48
Part I: Priority programme performance indicators (2-3 / sector) derived from the Indicator Guide;
Part II: A simple reporting format for PCA partners to report on a regular basis (i.e. monthly following sudden onset emergency, quarterly during chronic emergencies)
• The Addendum can be shared with potential UNICEF implementing partners in advance
to facilitate discussion / agreement on project monitoring
• The Addendum also includes a UNICEF implementing partner commitment to facilitate
Field Monitoring visits for UNICEF and/or their representatives. In addition it agrees that
UNICEF implementing partners will report to relevant Cluster/Sector Lead agencies and
to OCHA using the 3W reporting format.
What does it take to do it
• It is recommended that the PCA Addendum is prepared by CO monitoring sections
• The following template includes the recommended priority indicators from the Indicator
Guide.
• Non applicable Sections, Commitments and Benchmarks can be deleted.
• Refer to the Indicator Guide for alternative priority indicators
Potential Challenges Solutions Too many indicators in PCAs • Experience in piloting HPM shows that 2-3
indicators per sector is a realistic amount to expect in terms of regular data collection.
• UNICEF should focus on what information is Needed and can be Used.
• Too many indicator are a significant burden on partner data collection
Insufficient UNICEF capacity to monitor
• UNICEF Monitoring units should support programme sections in developing tools and analysing data
• UNICEF programme sections must hold their PCA partners accountable for programme reporting
• Additional monitoring capacity can be explored through recruiting and training additional short term monitoring staff or through a 3rd party organization
Staff mindsets: Different approach to monitoring
• HPM advocates a different approach to monitoring than the traditional UNICEF non-humanitarian approach
• This is necessary for UNICEF for upward accountability for funds administered; and downwards accountability that humanitarian programmes achieve the desired results for the targeted people
* Please share Challenges and Solutions through the Humanitarian PM Community of
Practice for inclusion in future drafts of the Humanitarian PM guide.
http://intranet.unicef.org/CoP/EMOPSHPM/CommunityContent.nsf
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
49
Part I – Priority programme result indicators by sector
• Delete as Programme Section / CCC Commitments Benchmark and Indicators as appropriate
Nutrition CCC Commitment CCC Benchmark High frequency Indicator Guidance notes
Children and women with acute malnutrition access appropriate acute malnutrition management
Effective management of acute malnutrition (recovery rate is >75%, and mortality rates are <10% in therapeutic care and <3% in supplementary care) reaches the majority of the target population (coverage >50% rural area, >70% urban area, >90% camp)
# and/or % children 6-59 months with SAM enrolled in TFP or community-based programmes or facilities
Numerator = # children 6-59 mo with SAM enrolled or admitted in TFP or community based programmes or facilities Denominator = estimated # children 6-59 mo with SAM based on most recent survey
# and/or % children 6-59 mo with MAM enrolled in supplementary feeding programmes
Numerator = # children 6-59 mo with MAM enrolled in supplementary feeding programmes Denominator = estimated # children 6-59 mo with MAM based on most recent survey
% of exits from targeted supplementary feeding programmes of children 6-59 months who have: - Died (should be less than 3%) - Recovered (should be more than 75%) and - Defaulted (should be less than 15%)
The time needed to achieve the exit indicators for a supplementary feeding programme is 1 to 2 months. This will be a more advanced stage of high frequency monitoring. Exits from a feeding programme are those no longer registered. The population of exited individuals is made up those who have defaulted, recovered (including those who are referred) and died. Those admitted after discharge from therapeutic care should be reported as a separate category. See SPHERE 2011 Food Security & Nutrition Standard1 Guidance Note 5 for details on measurement p.167
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
50
• Delete as Programme Section / CCC Commitments Benchmark and Indicators as appropriate
Health
CCC Commitment CCC Benchmark High frequency Indicator
Guidance notes
Children and women access lifesaving interventions through population-based/community-based activities e.g. campaigns and child health days
95% coverage with measles vaccine,
# and % of children 6 - 59 months and 6mo-15y vaccinated for measles
Numerator = # of children 6 - 59 months and 6mo-15y vaccinated for measles Denominator = estimated # of children 6 - 59 months and 6mo-15y in affected areas (adjust up where registration higher than expected)
All families in the affected area receive 2 insecticide-treated bed nets (ITNs) in malaria endemic areas
# and % of families receiving 2 ITNs
Numerator = # of families receiving 2 ITNs Denominator = estimated # of families in affected areas (adjusting as above)
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
51
• Delete as Programme Section / CCC Commitments Benchmark and Indicators as appropriate
WASH
CCC Commitment CCC Benchmark High frequency Indicator Guidance notes
Children and women access sufficient water of appropriate quality and quantity for drinking, cooking and maintaining personal hygiene
Children and women have access to at least 7.5 - 15 litres of clean water per day
# and/or % of population with access to 15 litres of water per person per day
Numerator = # of water sources for each type of water source X # of people to be served by each type of water source (see below) Denominator = # people in the affected area See # of people per source by yield: 250 people per tap based on flow of 7.5 L/min. 500 people per hand pump based on flow of 17 L/min 400 people per single user open well based on 12.5 L/min (See SPHERE Wather Supply Standard 1-Guidance Note 4 and 5, p. 199) Where water tankering is used numerator reqiures also adding #L delivered divided by 15 L.
Children and women access toilets and washing facilities that are culturally appropriate, secure, and sanitary, and are user friendly and gender appropriate
A maximum ratio of 20 people per hygienic toilet / latrine squat hole.
# and % of people living in faeces free environment AND # and % of people with access to appropriately designed toilets
Options as situation evolves (see also early input/process indicators): Numerator = # of communal toilets established for women + # communcal toilets established for men + # of family toilets X # of people targetted for each type of toilet (See SPHERE Excreta Disposal Standard 2-Guidance Note 1-4 for details on appropriate coverage p. 109) Denominator for all = # of people in affected areas Note: Assumptions must be agreed ideally at cluster level on what figure will be estimated for existing % of people with access to toilets or living in a faeces free environment based on pre-crisis practices and toilet/atrine coverage and how this has changed from pre-crisis contexts. Also care must be taken to avoid double counting where improved latrines are being constructed to replace immediate options such as trench latrines. This does not increase % living in faeces free environment. For this reason, it is important to
track both indicators.
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
52
• Delete as Programme Section / CCC Commitments Benchmark and Indicators as appropriate
Child Protection CCC Commitment CCC Benchmark High frequency Indicator Guidance notes
CP2. Monitoring and reporting on grave violations and other serious protection concerns for children and women is undertaken and systematically triggers response (including advocacy)
2: Periodic reports are available and utilized on grave violations and other serious protection concerns for children and women
Achievement of milestones toward established of monitoring and reporting (qualitative)
AND/OR
% of population covered by monitoring systems
Milestones include: - active monitoring and reporting on grave violations against children and other serious child protection concerns; - measurable and verified data exists on grave violations and other serious child protection concerns (by violation); - periodic reports available; - development and implementation of prevention and response mechanisms; - systematic data is available on grave violations and other serious child protection concerns over time.
CP3. Key child protection mechanisms are strengthened in emergency-affected areas
Safe environments are established for the most vulnerable children
# and % of children reached through child-friendly spaces in displacement sites and camps
'Numerator=# of children enrolled in Child Friendly Spaces Denominator=# of children agreed as targeted "most vulnerable" overall in affected area. This may for example be all displaced children. A decision Is required on targeting. Field monitoring is required to explore quality issues of CF spaces, including success of cross-sectoral coordination of services.
CP4. Separation of children from families is prevented and addressed and family-based care is promoted.
All separated and unaccompanied children are identified and are in family-based care or an appropriate alternative
# of separated and unaccompanied children identified.
'- while denominators are estimates for all indicators in emergency settings, denominators for separated children are especially unclear, i.e. unknown until children are identified and at the same time are integrated in programmes. This is one of few indicators where reporting the simple number is considered adequate and meaningful
# and % of separated children in emergencies reunified
Numerator = # of separated children reunified Denominator = total # of separated identified since beginning of crisis or, for ongoing situations, in current year
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
53
CCC Commitment CCC Benchmark High frequency Indicator Guidance notes
CP5. Violence, exploitation and abuse of children and women is prevented and addressed, including GBV
5: Affected communities are mobilized to prevent and address violence, exploitation and abuse of children and women: existing systems to respond to the needs of GBV survivors are improved
# and % of population in communities where ongoing work to mobilize and strengthen social support networks to prevent and address violence, abuse and exploitation, including GBV
Numerator = # of communities where ongoing work to mobilize and strengthen social support networks to prevent and address violence, abuse and exploitation, including GBV x # population in each community Denominator = total affected population
CP6. Psychosocial support is provided to children and their caregivers
6: All CP programmes integrate PSS in their work in line with the IASC MHPSS guidelines
# and % children enrolled in psycho-social activities
Numerator = # children enrolled in CP programmes that have integrated IASC MHPSS guidelines Denominator = total child population (for target age group if strategy is age targeted) Important that this is supported by systematic field monitoring to ensure quality of adherence to MHPSS guidelines.
CP7. Child recruitment and use; illegal and arbitrary detention is prevented and addressed for conflict-affected children
7: Inter-agency plan developed and implemented for prevention and response to child recruitment; advocacy against illegal and arbitrary detention for conflict-affected children is conducted.
# of children newly released from armed forces and groups
'-Newly' means since last reporting period; both cumulative and newly released figures can be presented - No % is reported as denominator is especially unclear - this serves as a proxy indicator for success in advocacy
# of children reached through response and reintegration programmes and services.
'- while denominators are estimates for all indicators in emergency settings, denominators for CAFFare especially unclear even where MRM established. This is one of few indicators where reporting the simple number is considered adequate and meaningful
# and % of children reintegrated into their families and communities.
Numerator = # of children male/female reintegrated Denominator = # of children reached through response and reintegration programmes and services.
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
54
CCC Commitment CCC Benchmark High frequency Indicator Guidance notes
CP8. The use of landmines and other indiscriminate or illicit weapons by state and non-state actors is prevented and their impact addressed
8: Children and communities in affected areas have access to mine/UXO risk education and are better protected from the effects of landmines and other indiscriminate and/or illicit weapons
'# and % population exposed to mine/UXO risk education activities
'Requires detailed monitoring by type of activity and either estimated coverage by type or reported coverage. Depends on C4D strategy developed. See separate guidance on monitoring C4D activities
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
55
• Delete as Programme Section / CCC Commitments Benchmark and Indicators as appropriate
Education CCC Commitment CCC Benchmark High frequency Indicator Guidance notes
E2. Children including preschool age children, girls, and other excluded children, access quality education opportunities
Schools re-opened and child-friendly and adolescent-friendly emergency non-formal programmes, including play and early learning for young children, established for affected communities
# and % of school-aged children including adolescents reached by schools (including in schools in affected areas still functioning, re-opened schools and/or temporary facilities established)
Numerator = # schools still functioning + # schools re-opened + # temporary facilities X estimated average # of children per type of facility agreed at cluster level
Denominator = total # school aged children in affected area (aged 4-6 years as target for pre-primary; aged 6-14 as target for basic; aged 15-18 as target for post-basic)
Option to disaggregate by age group covered in both numerator and denominator
Estimated average # of children should be periodically cross-checked through field monitoring
# and % of school-aged
children including adolescents
reached by child-friendly and
adolescent-friendly emergency
non-formal education
programmes, including play and
early learning for young
children
% made up of # non-programmes still functioning added to # re-opened or new multipled by the estimated average # of children per facility agreed at cluster level per total # school-aged children in affected area
Estimated average # of children should be periodically cross-checked through field monitoring
Important in reporting that child-friendly education programmes are clearly distinguished from protection- or health- oriented CFS to ensure no double counting and avoid confusion by data users
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
56
CCC Commitment CCC Benchmark High frequency Indicator Guidance notes
E4. Psychosocial and health services for children and teachers is integrated in educational response
All education humanitarian response integrates appropriate psychosocial, health and nutrition interventions
# and % of children with access to Humanitarian Education programmes that incorporate psychosocial support
[Note: NATF indicator: % of schools/learning spaces offering psychosocial support for (a) children and youth; (b) teachers (E8)]
Option 1:
Numerator = # of educational programmes where psychosocial programming is integrated (by type of programme) X average # children by type of programme (agreed at cluster level)
Denominator = total # of school-aged children in affected area (aged 4-6 years as target for pre-primary; aged 6-14 as target for basic; aged 15-18 as target for post-basic)
Estimated average # of children should be periodically cross-checked through field monitoring
Option 2:
Numerator = # of teachers trained to agreed cluster or national standard for adequate knowledge, skills and good practices for supporting children's pychosocial needs X average # children reached by teacher (agreed at cluster level)
Denominator = as above
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
57
• Delete as Programme Section / CCC Commitments Benchmark and Indicators as appropriate
HIV/AIDS
CCC Commitment CCC Benchmark High frequency Indicator
Guidance notes
HIV2. Children, young people and women access HIV and AIDS prevention, care and treatment during crisis
2: 80% of emergency affected population access relevant HIV and AIDS prevention, care and treatment services, e.g. post rape care including Post-Exposure Prophylaxis (PEP), Sexually Transmitted Infections (STI) treatment, prevention of mother-to-child transmission (PMTCT) and antiretroviral (ART)
# women, young people and children victims of sexual violence provided with PEP
'This is an area where denominators are especially problematic therefore the raw # served is used in monitoring. This should not be pursued if there are any concerns about safety of data collection in terms of negative repercussions for victims. Disaggregation of data collection by age is recommended to use >18; 10-18 and <10. Qualitative field monitoring should include verification if there are any instances of young children being turned away or referred to PMTC clinics due to concern about appropriate dosage
HIV3. Prevention, care and treatment services for children, young people, and women are continued
3: 80% of emergency affected population previously on HIV related care treatment continue to receive ARVs for prevention of mother-to-child transmission (PMTCT) and, children and young people on antiretroviral treatment (ART) continue receiving ART.
# and % HIV positive pregnant women continuing to receive ARVs for PMTCT
Option 1 recommended where, as per normal practice in emergencies, new PMTCT is not initiated : Numerator = # of HIV positive pregnant women receiving ARVs for PMTCT Denominator = # previously receiving ARVs for PMTCT Option 2 recommended where monitoring more advanced with patient identification cards allowing tracking of: - # new patients - # new from other locations - # continuing patients - # lost to follow-up In such cases both # and% continuing can be tracked as above as well as # and % new cases
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
58
Part II – Simplified Reporting format for Monthly PCA progress reporting
UNICEF and _________________ Civil Society Organization / Partner
PCA Reference No.___________________________
Reporting Period: From______________to ______________ Project Duration: From__________________ to _______________________
% Project Duration Passed__________________ Total UNICEF Contribution________________ % Project funding spent____________________
Expected Result /
Outcome Progress Indicator Cumulative Result to date
Issues / Constraints /
Comments Necessary Actions
UNICEF PCA partners agree to facilitate Field Monitoring visits for UNICEF and/or their representatives.
UNICEF PCA partners agree to report against the OCHA 3W template
UNICEF PCA partners agree to report according to relevant Cluster/Sector Lead agencies according to the framework and mechanism agreed
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
59
UNICEF PCA / Intervention Mapping
Q&A Guide on Using Tool
Contents of this section
• Why is this important (what the Country Management Team should know)
• What users should know
• What does it take to do it
• PCA Mapping template
Why is this important (what the Country Management Team should know)
• The rapid scale up of UNICEF interventions following a sudden onset emergency has a
number of risks:
o Programme design risks of ‘doing harm’ in complex contexts and rapid scale-up;
o Financial and implementation management risks when there is sudden increase in
resources and UNICEF implementing partners32;
o Reputational risks where UNICEF is unable to say what the progress is in reaching
affected population, or the progress in country cluster/sector coordination where lead
agency
• In this context the opportunities for harmonized cross-sector thematic programmes and
administration efficiencies of partner agreements can easily be missed
• This tool seeks to fill any gap in UNICEF CO intervention mapping
• The initial mapping exercise is likely to be timely and clear tasking is required.
• Unless the mechanism for intervention mapping already exists – it is proposed that it is
carried out by the CO Monitoring section
What users should know
• Unless the mechanism for intervention mapping already exists – it is proposed that it is
carried out by the CO Monitoring section
• CO Programme section chiefs are likely to know where interventions are within their own
sector - In this case a collation exercise is required to bring all sector interventions
together in a single document or map
• Where there is a significant scale up of operations and rapid increase in interventions it
will be useful to map UNICEF interventions to allow opportunities for harmonized cross-
sector thematic programmes and administration efficiencies of partner agreements to be
utilized
What does it take to do it
• A map, some coloured pins and a camera (to photograph map and share on e-mail) is an
appropriate alternative to hiring a GIS specialist and converting the data into a database
• Mapping software is useful, however if it is not already well established in the CO then
serious consideration should be given to the cost to establish versus potential benefits
• If GPS units are available then interventions be plotted onto Google maps
32
See UN Harmonized Approach to Cash Transfers (HACT) for resources on reducing financial risk
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
60
Intervention mapping template – Provincial, Country example, Pakistan 2010 flood response (extract from Xcel document)
Date: 10-07-10
S# Section
PCA/SSA/SSFA
No.
PCCA/SSA/S
SFA Start Date End Date
Total UNICEF
Share PKR Name Partner Govt/NGO
Partner;s Address
Contact Details
Focal Point
(Partner)
Focal Point
Contact Details
Location of
Interventions
(District/UCs) GPS Coordinates Major Activities Benfciaries
1 Health PAK/10/124 PCA 01-Aug-10 31-Dec-10 21,418,654 RI NGO Matta, Swat
Community Mobilization,
OTP,
Stabilization Centre,
388,954
2 Health PAK/2010/124 PCA 01-Jul-10 31-Dec-10 23,545,983 CERD NGOJalozai IDP Camp
Tough Sarai Camp Hangu
Provision of access to PHC at Tough Sarai Camp Hangu
Provision of access to MHC at Tough Sarai Camp Hangu
Provision of CMAM services at Tough Sarai Camp Hangu
Provision of access to MHC at Jalozai IDP Camp
Provision of CMAM services at Jalozai IDP Camp
105,000
3 Health PAK/10/125 PCA 01-Jul-10 31-Dec-10 22,477,202 MERLIN NGO Jalozai IDP CampProvision of CMAM services at Jalozai IDP Camp
106,000
4 Health PAK/10/169 PCA 01-Sep-10 31-Dec-10 15,876,486Abaseen
FoundationNGO
Civil Hospital Kabal Swat
and its linked 8 health
facilities
Provide immediate access to Women and Children to Static
life interventions
Communities have improved access to quality health services
through community organization and participation.
Community awareness through standard health messages
Development of Public Private Partnership Agreement
225,000
5 Health PAK/10/149 PCA 16-Aug-10 15-Oct-10 3,960,200 PPA Semi Govt Nowshera & CharsaddaProvision access to Children to Child Health Care services
Manage Acute Watery Diarrhea per IMCI protocol120,000
6 Health PAK/10/148 PCA 16-Aug-10 31-Dec-10 21,127,141 NRSP NGO 12 Ucs of Nowshera
Provision of Adequate nutritional care and treatment for
children, pregnant women and lactating women.
Prevetion of malnutrition in early childhood.
Revention of micro nutrient defeciencies in children
To strengthen local tehnical capacity
327,897
7 Health PAK/10/77 PCA 01-May-10 31-Oct-10 13,638,268
The Johanniter
International
Assistance
NGO 6 affected Ucs of Kohat Emergency nutritional interventions for IDPs 171,781
8 Health PAK/10/78 PCA 01-May-10 31-Oct-10 13,604,748 CDO NGO 6 affected Ucs of Hangu Emergency nutritional interventions for IDPs 144,000
9 Health PAK/10/79 PCA 01-May-10 31-Oct-10 16,087,801 PHRP NGO 8 affected Ucs of KohatProvision of Quality Nutrition services to IDPs and host
population192,000
10 Health PAK/10/80 PCA 01-May-10 31-Dec-10 21,495,496 FPHC NGO8 affected facilities at
Mohmand Agency
Provision of Quality Nutrition services to returnee IDPs and
stayed back committees224,000
11 Health PAK/10/81 PCA 01-May-10 31-Dec-10 26,806,485 RI NGO12 affected UCs at
District Lower DirCommunity Based Management of Acute Nutrition 299,600
Planning , Monitoring and Evaluation (PME), UNICEF Peshawar, Details of PCAs/SSAs/SSFAs
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
61
Field Monitoring
Q&A Guide on Using Tool
Contents of this section
• Why is this important (what the Country Management Team should know)
• What does it take to do it
• Options to consider when applying
• Field Monitor Visit Checklist - tool
A) Input Monitoring
B) Cooperation Partner progress report verification
C) Qualitative Monitoring and Observation
• Guide for Using the Field Monitor Checklist - tool
o Field Monitor deployment plan
o Weekly Field Monitor Summary Report
• Contracting Field Monitors – Template for SSA with Organization
• Contracting Field Monitors – Template for Individual staff SSA
Why is this important (what the Country Management Team should know)
• “Field Monitoring” is used to describe systematic visits to UNICEF projects or where
UNICEF supplies are distributed, which are carried out by UNICEF staff or their
representatives.
• Field Monitoring visits give UNICEF an essential reality check of what is happening on the
ground in environments where much can go wrong. It involves the affected population in
monitoring the UNICEF response.
• CO senior management needs to consider their current capacity to carry out Field Monitoring visits and decide whether to increase CO capacity and how.
• The Humanitarian PM Field Monitoring approach proposes that UNICEF COs recruit
additional capacity through a 3rd party organization or through direct hire of Field Monitors
• Field Monitors are not intended to replace technical visits by UNICEF programme staff but
to ‘put more feet on the ground’ to support higher frequency humanitarian monitoring
• To be useful, field monitoring must be higher coverage/frequency (more ‘feet on the
ground’) and needs to be a structured and systematic approach with clear criteria on Where
to go and What to verify. It should utilize CO PCA mapping
• The Humanitarian PM toolkit proposes three objectives for Field Monitoring visits
Field Monitoring Visit Objective How a) Monitor the status of delivery of UNICEF supplies
beyond leaving the UNICEF warehouse (reducing financial risk to UNICEF)
Checking status of UNICEF inputs based on Supply Distribution Lists
b) Cross-check PCA partners progress reports (reducing financial risk to UNICEF)
Verifying cooperation partner progress reporting based on latest PCA partner reports
c) Inclusion of the affected population, that interventions are appropriate, identify any areas of exclusion and that UNICEF interventions ‘Do No Harm’ (through interviews and observations)
Through observation and discussions with end-users at UNICEF intervention sites
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
62
• Parts a) and b) are generic, however part c) will require adapting to the specific country
context to reflect local priorities and issues – this is likely to require adapting to specific sub-
national contexts.
• The PCA Monitoring and Reporting Addendum tool supports the Field Monitoring approach
by agreeing a higher frequency of lighter progress reporting based on a few prioritized
indicators for each section.
What does it take to do it
• Scaling up UNICEF monitoring capacity requires
a) Resources: funds for Field Monitor contract, ideally GPS units
b) Clear tasking within CO, i.e. which unit will manage the Field Monitors (recommend the
CO Monitoring unit where feasible)
c) Clear tasking in terms expected outputs from Field Monitors
• A 3rd party research organization can be hired through a single SSA.
• Individual Field Monitors can be hired through staff SSAs
• As part of pre-emergency planning suitable 3rd party organizations can be identified and
budgeted for.
Options to consider when applying
CO capacity for carrying out monitoring visits
• In stable contexts Field Monitoring visits are carried out by the relevant Programme Section
and their PCA partners.
• Following a sudden scale up of operations, or where access is an issue, additional
monitoring capacity will be required.
• Options for scaling up UNICEF monitoring capacity include,
a) Hiring of a 3rd party organization to manage a team of Field Monitors on UNICEF’s
behalf (managed by the CO Monitoring unit), OR;
b) Direct hiring and training of UNICEF field monitors (lead by the CO Monitoring unit)33
• Field Monitoring visits can begin when UNICEF inputs start being distributed.
Training Field Monitors
• The level of training and specialization of Field Monitors will vary from country to country
• Field Monitors may not be programme specialists and regular training and familiarization with UNICEF programme staff is likely to be necessary.
• Where feasible specific training (i.e. Participatory Rural Appraisal, Rapid Rural Appraisal) should be carried out for Field Monitoring34.
33 Where a suitable organization exists it is recommended that COs hire an organization. Hiring an organization may be more expensive in financial cost,
however hiring and managing 10 or 15 Field Monitors each on individual SSA
contracts, organizing transport, DSA etc. has significant hidden costs in time, transport, administration etc. 34
There is an abundance of resources on PRA/RRA resources and how to undertake them. For an overview with strengths and weaknesses of the approach see the Oxfam Development Handbook:
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
63
Managing Field Monitors • While Field Monitors provide additional generic capacity to monitor, they should not replace
visits by UNICEF programme staff who provide technical oversight.
• Field Monitoring visits should be based on UNICEF supply distribution (for the initial weeks) and then be a mix of UNICEF supply distribution and cooperation agreement project sites through UNICEF PCA mapping.
• As much as possible Field Monitor visit should be coordinated with as many UNICEF sections as possible.
• Sections (A) and (B) of the Field Monitoring visit checklist are generic and apply to UNICEF Supply Distribution lists and Cooperation Partner reports
• Section (C) is programme and context specific. The checklist provides some key themes to guide observations and discussions however they should be adapted to the local context to ensure information collected is usable to UNICEF.
• The exact format of Section (C) is flexible from country to country, as well as within countries.
• Continuous revision of Section (C) of the Checklist is critical to remain focused on the real-time information needs for UNICEF programme sections. Propose that the Checklist is reviewed every 2-4 weeks with CO monitoring section and Programme staff
Information Management
• Information collected by Field Monitors need to remain useful to the UNICEF and sector response. Regular feedback meetings with UNICEF programme staff and retaining a flexible approach can ensure that information collected can be utilized.
• Field Monitoring reports should be regular (i.e. weekly) and be based on UNICEF intervention PCA mapping.
• As a guide the role of the Field Monitors should be to flag issues for programme colleagues. Programme sections determine what, if any, action to take.
• A “database” for Field Monitoring information is unlikely to be necessary unless one already exists and can be easily updated. Field Monitoring information can compliment PCA mapping (GIS mapping capacity is not necessary)
Potential Challenges Solutions
Who manages Field Monitors in UNICEF
• Best suited to the CO Monitoring Unit
• ToR for 3rd
party organization includes Field Monitoring Coordinator to manage day-to-day and collate reports
What to do with Field Monitoring information
• FM information should be shared primarily with CO programme staff who decide what, if any, action is necessary (programme decision)
• Information relevant to the wider cluster should be shared through cluster meetings
• FM information is important both for the specifics (a serious problem in a given site) and the overall trend. Aggregating data is important. (Note that a simple database that allows UNICEF and partners to input, share and analyse data is under development.)
• If FM information is not useful to UNICEF programme sections then the collection of the information should be reviewed and adjusted in the checklist
Overlapping Field Monitoring initiatives by multiple agencies
• UNICEF should share that it is carrying out FM visits with OCHA and others
Sensitivities when monitoring government partners
• If FMs are planned for use with government partners then some outreach work may be necessary to ensure government partners are aware and facilitate FM visits
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
64
Examples of challenges and proposed solutions from Pakistan flood response 2010
Challenges Proposed Solution
Punjab
PCA partners have started viewing FMs as ‘watchmen’ and develop tendency to hide information. Similar feelings can develop amongst programme staff as well
The role of FMs need to be defined in PCAs and MOUs to inform PCA partners
Request for use of FMs for validation, use in surveys/ assessments and other similar assignment
The FMs should not be involved in tasks beyond their capacity.
Balcohistan
PCA partners have a poor response to FMs UNICEF POs to ensure PCA partners are aware of the role of FMs The role of FMs need to be defined in PCAs and MOUs to inform PCA partners
Beneficiary (receipt of supplies) monitoring –people have the tendancy of saying that they did not receive anything regardless
• Random visits to differnet households in a single locality
• Cross checking
• Observation
Sindh
Sustaining remote field monitoring after the contract ends
Incorporate budget for field monitoring as part of program cost
Limited sector specific technical knowledge of Field Monitors
Regular meeting with programme sections and sharing of relevant documents (description/pictures of distribution materials and definitions of technical terms) and joint field monitoring visits
PCA partners poor response to Field Monitors A letter to be sent from UNICEF to all PCA partners informing that UNICEF assisted programs are subject to monitoring. This would be in addition to the clause on monitoring in the PCA.
KP
FM have limited sector specific technical knowledge of field monitors
Regular meeting with programme sections and sharing to relevant documents (description of distribution materials and definitions of technical terms)
* Please share Challenges and Solutions through the Humanitarian PM Community of
Practice for inclusion in future drafts of the Humanitarian PM guide.
http://intranet.unicef.org/CoP/EMOPSHPM/CommunityContent.nsf
Humanitarian Performance Monitoring Toolkit and Guide 01June2011
65
Field Monitor Tools
Field Monitor Visit Checklist
A) Input monitoring
B) Cooperation Partner progress report verification
C) Qualitative Monitoring and Observation
Field Monitors should complete the sections of this Checklist, continuing on additional sheets as
necessary. One sheet should be completed for each location that is visited. A weekly summary of
visits should be prepared and shared with UNICEF Monitoring focal point.
A) Input Monitoring
When: Can be carried out as soon as UNICEF inputs are delivered to partners / affected population
Objective: To identify any bottle-necks in supply of UNICEF inputs after delivery to cooperation
partner or affected population
Name of Site Visited: Date of Visit:
Programme Section: Cooperation Partner(s):
Location (sub-district/district/Province: GPS:
Items Provided Field Monitors Comments
• Details of Items delivered should be obtained from the Supply Distribution List from the relevant
Country Office Programme Section and/or Supply and Logistics Unit through UniTrack
• If there are too many items on Supply Distribution List then Field Monitors should pick a random
sample, i.e. every 2nd
items, or every 3rd
item to check
• Continue of additional sheets as necessary
Humanitarian Performance Monitoring Toolkit and Guide 01June2011
66
B) Cooperation Partner progress report verification
When: As soon as UNICEF cooperation partners begin submitting progress reports, typically one
month after initial signing of PCA including the PCA Monitoring and Reporting Addendum.
Objective: To verify cooperation partner progress reports, in coordination with cooperation partners
and in place/ or in conjunction with UNICEF programme section field monitoring visits.
Name of Site Visited: Date of Visit:
Programme Section: Cooperation Partner(s):
PCA Number: Cooperation Partner report dated:
Location (sub-district/district/Province: GPS:
Progress Reported Field Monitors Comments
• Details of Items delivered should be obtained from the Cooperation Agreement Progress Report
from the relevant Country Office Programme Section. For example of the Progress Report see the
PCA Monitoring and Reporting Addendum tool from the Humanitarian PM toolkit.
• If there are too many outputs to monitor then Field Monitors should pick a random sample, i.e.
every 2nd
items, or every 3rd
item to check
• Continue of additional sheets as necessary
Humanitarian Performance Monitoring Toolkit and Guide 01June2011
67
C) Qualitative Monitoring and Observation
When: Can be carried out as soon as UNICEF inputs are delivered to affected population and
UNICEF cooperation agreement projects begin.
Before use this section requires reviewed, adapted and validated at the country office level by
programme sections. Where using at the provincial or sub-provincial level it should be adapted to the
local context. This may mean a different qualitative section for different provinces, sub-provinces.
Objective: To asses suitability of UNICEF project approach through observation and interviews with
recipients and end users.
Name of Site Visited: Date of Visit:
Programme Section: Cooperation Partner(s):
Location (sub-district/district/Province): GPS:
Nutrition
OTP (Outpatient Therapeutic Programme) Stabilization Centre
Cooperation Partner Discussion Cooperation Partner Discussion
Is the OTP working o Yes o No
Is the SC working
o Yes o No
Is there counselling on Infant and Young Child Feeding
o Yes o No
Is a Doctor / Health worker available at the SC
o Yes o No
How many acute, malnourished, children have been omitted in the centre reporting period
Number ____________ Report period___________
Is there counselling on Infant and Young Child Feeding
o Yes o No
What is the recovery rate of the treatment centre
___________% What is the recovery rate of the treatment centre
___________%
What is the medical and food stock
Food type Days of stock ________ ________ ________ ________ ________ ________ ________ ________
Does the centre have sufficient medical and food stock
Food type Days of stock ________ ________ ________ ________ ________ ________ ________ ________
Discussion
When was the last distribution of nutrition kits
When was the last distribution of high emergency biscuits
Who in the family uses which items in the nutrition kit
Can people demonstrate they know key UNICEF messages
Is anyone excluded from services, if yes, Who, Why and how many
Are there community based facilities to serve malnourished children
Humanitarian Performance Monitoring Toolkit and Guide 01June2011
68
Health
Discussion
How many days since the last visit by the Community Health Worker (CHW)
How long did you wait the last time you saw the CHW
To pregnant / new mothers, did you receive any antenatal/postnatal care from health care provider?
o Yes o No
Can people demonstrate they know key UNICEF messages
Is support available for delivery
o Yes o No
Are delivery kits available o Yes o No
If CHW in the community, are stocks sufficient, if not – what is missing
Is anyone excluded from services, if yes, Who, Why and how many
Are locally based health workers trained in Health Service Systems, if Yes, How many, How recent
What is the role of local government authorities involved in providing services?
WASH
Observation Discussion
What are the main sources of drinking water
How much water is used per person per day (estimate use from number and size of container used)
Litre Per person per day
How does community dispose of excreta
How far do people walk to access water
Is there open defecation? Where Aqua tab and pure Sachet, does the community know how to use and mix
Where there are latrines: How many people use how many latrines
Does everyone access latrines - if not why not
Are latrines in use – if not why not?
Is community participative in operation of latrines – if yes How
How is garbage disposed Have families in the community received hygiene kits
Did the site appear clean, if not Why
Who in the family uses which items in the hygiene kit
Is there bathing facilities available in the community with separate facilities for women
What is the role of local government authorities involved in providing services?
Is anyone excluded from services, if yes, Who, Why and how many
Can people demonstrate they know key UNICEF messages
Humanitarian Performance Monitoring Toolkit and Guide 01June2011
69
Child Protection
Observation Discussion with community / Cooperation Partner
How many Child Friendly Spaces (CFS) and how many children using them
How many children are registered/attending CFS (by sex, by age)
How many care givers are there at a single CFS?
What are the criteria for registration in CFS?
Is there any recreational activities going in CFS, if yes what type
Are there any children not yet reunited? How many
Which basic services are available at CFS? (water, sanitation, safety, hygiene)?
What actions are taken to reunite children with their family?
Do children have safe space in the community to play
Is there Psycho social support for children, if Yes in what form
Are children involved in labour? If yes, what type of labour
Can people demonstrate they know key UNICEF messages
Are children involved with armed groups
Are referral services available at CFS?
Is anyone excluded from services, if yes, Who, Why and how many
Education
Observation Discussion
Where does teaching take place
Are any children excluded from services, if yes, Why and how many
Is school in the community operating
Are there enough school supplies
Is the learning facility damaged, how much
Is there drinking water and toilet facilities at teaching places
Do boys and girls access toilet facilities, if not Why
Do they wash their hands after using toilet, if not Why
Where there is a TLC Is TLC functional, if Not why?
Where there is a TLC What are the criteria used to enrol children in TLC?
What are the basic services (water, hygiene) available at TLC?
Who is teaches students at TLC?
How many children use the TLC (boys and girls)
Can children demonstrate they know key UNICEF messages
What is the role of local government authorities involved in providing services?
Humanitarian Performance Monitoring Toolkit and Guide 01June2011
70
Guide for Using the Field Monitor Checklist
• This guide supports the UNICEF section / unit / staff in managing the use of the Field
Monitoring Checklists and compliments the Field Monitoring ToR35.
• The guide and checklist can also be used by UNICEF programme section staff as part of
their regular UNICEF project visits
• The guide is intended for use by UNICEF CO to plan and manage the organization
responsible for the Field Monitors – or individual Field Monitors contracted by UNICEF -
and provides clarity on the expected information outputs to support the UNICEF and
UNICEF led cluster/sector response.
• To avoid spending too much time travelling, and to maximise time with affected
population Field Monitors should aim to spend one day per location. This can be
adjusted to suit the local context.
1. Objective of Field Monitors
• Field Monitors have been contracted to complement input and output monitoring
undertaken by UNICEF programme officers. Field Monitors DO NOT replace Remote
monitors or technical oversight visits by UNICEF programme staff and should not be
tasked with collecting the same information.
• As a guide additional field monitors provide extra ‘feet on the ground’ for UNICEF –
providing capacity to visit UNICEF projects where access is an issue, or where a rapid
scale up of operations means the CO does not have the capacity to monitor
• The objective of Field Monitors should be adapted to suit the country context, as a guide
the following areas are recommended for focus:
a) Monitoring UNICEF supplies (beginning from when UNICEF supplies are distributed)
b) Cross-check PCA partners progress reports (beginning when UNICEF partner reports
are submitted – typically one month after the project start date36
c) Observations and Discussions with the affected population (beginning when UNICEF
supplies are distributed)
d) Special monitoring task (as and when necessary, determined by CO)
2. Approach
• Field Monitors will use various approaches to collect information. The Emphasis is on
qualitative data
• Enumerators are NOT collecting information for a quantitative survey. Provincial PME
should be clear on the Field Monitoring Outputs expected with the contracted
organization. We are NOT looking for statistical significant data for generalisation across
a certain population group
Field Monitoring Visit Objective How a) Monitor the status of delivery of UNICEF supplies beyond leaving the UNICEF warehouse (reducing financial risk to UNICEF)
Checking status of UNICEF inputs based on Supply Distribution Lists
b) Cross-check PCA partners progress reports (reducing financial risk to UNICEF)
Verifying cooperation partner progress reporting based on latest PCA partner
35
The Field Monitor ToR for the contract includes the provision that Field Monitors should be made up of equal numbers of men and women and 2 person teams should include one male and one female 36
This is facilitated by the use of the PCA Monitoring and Reporting Addendum – Humanitarian PM toolkit
Humanitarian Performance Monitoring Toolkit and Guide 01June2011
71
reports
c) Discussions with the affected population, that interventions are appropriate, identify any areas of exclusion and that UNICEF interventions ‘Do No Harm’ (through interviews and observations)
Through observation and discussions with end-users at UNICEF intervention sites
a) Monitoring UNICEF supplies
• Will require Supply distribution lists from both CO Supply and Logistics and Programme
sections depending on supply management arrangements at CO
• Verify delivery of UNICEF supplies to end-users (i.e. tracking of UNICEF supplies from
UNICEF warehouse to the end user, identifying any delays, bottle necks and undelivered
supplies)
b) Cross-check PCA partners progress reports
• UNICEF PCA partners should be reporting on the simplified, higher frequency reporting
format contained in the PCA Monitoring and Reporting Addendum on a monthly basis.
• Verify with PCA partners, affected population and through observation the reported
progress
c) Observations and Discussions with the affected population
• There are a number of methods for discussion with affected populations.
• The CO will determined the most appropriate method for which groups.
The following table are some suggestions for consideration
Subject Information Source Utility of intervention Key informant interviews (with service providers)
Focus group discussions with users (All; women; children)
Access / Exclusion Key informant interviews (with women; children; marginalized groups; dissabled)
Safety Key informant interviews (with women; children)
Role of the government Observation, Key informant interviews (with local government authorities), Focus group discussions with all users
Do No Harm Focus group discussions with users (all, women, children)
Observation Walk through community
• Through use of part c) the checklist (adapted to the country context) identify any
issues/difficulties with regard to the UNICEF programme response (WASH, Health,
Nutrition, Education, Child Protection)
• Identify any urgent unmet needs of women and children in the affected areas
d) Any special assignments
• On a case-by-case basis undertake any special assignments required by the
management in regards to quick assessments, in-depth analysis of specific case/s
Humanitarian Performance Monitoring Toolkit and Guide 01June2011
72
• E.g. verification of particular PCA partner report, investigate any reports of misuse of
UNICEF supply inputs etc.
3. Expected Outputs of Field Monitoring
• Consolidated weekly report with results of Field Monitor visits presented by objectives;
location and UNICEF intervention.
• See sample reporting template, Annex A
Propose information is presented through
• Weekly Presentation of Field Monitor Findings: Organization focal point to Provincial
PME
• Weekly presentation by Provincial PME to Provincial Management meeting (including
Program Sections and Provincial Cluster Coordinators)
• Every one or two week - Presentation of Field Monitor Findings from Field Monitors to
UNICEF Programme sections
4. Step by Step
STEP (i) – Prior to departure
• UNICEF (recommend monitoring unit in coordination with Programme sections) to
formulate a plan of which UNICEF intervention sites will be visited in the following week.
NB. Requires the CO to have some form of PCA/UNICEF intervention mapping by
location. See Annex A for example of Field Monitor deployment plan
• Field Monitors should have the following information prior to departure and UNICEF
(recommend monitoring unit) should ensure it is available:
Objective Pre-departure information required
From
Location of UNICEF intervention to be visited
Suitable information to locate (ideally by GPS reference) Name of Implementing partner
PCA Mapping, Or Programme Sections
a) Monitor the status of delivery of UNICEF supplies
Supply Distribution Lists
Programme Sections Supply Section
b) Cross-check PCA partners progress reports (reducing financial risk to UNICEF)
Latest PCA partner report
Programme Sections
c) Discussions with the affected population, that interventions are appropriate, identify any areas of exclusion and that UNICEF interventions ‘Do No Harm’ (through interviews and observations)
Humanitarian Performance Monitoring Toolkit and Guide 01June2011
73
STEP (ii) – Upon Arrival at Community/Site
Walk through community (utility in winter? - alternative)
• Observe areas relating to checklist part c)
• Map available public service (water, health, school) – simple drawing
a) Monitoring UNICEF supplies
• Using Supply distribution lists - verify delivery of UNICEF supplies to affected population
• Note any bottlebecks (i.e. supplies in UNICEF partner warehouses) and the reason
b) Cross-check PCA partners progress reports
• Using UNICEF PCA partner report - verify with partners the progress detailed in the
report
• Note any delays, bottlenecks, access/exclusion issues (marginalized groups), safety
issues for women and children
c) Observations and Discussions with the affected population
Identify 4-5 households randomly (near road, away from road, poor, less poor)
• Using checklist part c) as a guide to key informant interviews
• As a guide do not use more than 1 hour with each household.
Arrange group discussions with key groups based on checklist part c)
STEP (iii) – Completion of Checklists
• Complete the checklist, using additional paper as necessary
• Field Monitor coordinators (either a specific person recruited directly, or part of the 3rd
party research organization contract) to summarise information, see guide to reporting
template Annex B.
• Summarise your general observations and note the gaps in both UNICEF’s project
response and the situation for women and children
5. Review of Process
• Continuous revision of the Checklist is critical to remain focused on the real-time
information needs for UNICEF programme sections
• Propose that the Checklist is reviewed every 2 weeks with relevant programme staff
Provincial PME
Humanitarian Performance Monitoring Toolkit and Guide 01June2011
74
Field Monitor deployment plan
Last Updatd on: Oct. 08, 2010
Monitor Name Contact No Districts Assigned Partners working 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Faiz Muhammad 0333 648 4659 Jacobabad
HELP, Save the
Children &
HANDS
Meeting
with
unicef
Meeting
with
unicef
Nida Barohi 0332 285 6545 Shikarpur
HELP, Save the
Children &
HANDS
Meeting
with
unicef
Meeting
with
unicef
Ghulam Murtaza 0334 2904670 Sukkur
HANDS, Save the
Children, World
Vision, GSF
Meeting
with
unicef
Meeting
with
unicef
Maria Nawaz 0331 929 6527 KhairpurHANDS, World
Vision
Meeting
with
unicef
Meeting
with
unicef
Imran Memon 0333 727 2096 Larkana GOAL
Meeting
with
unicef
Meeting
with
unicef
Qamber ShahdadkotHELP, HANDS &
Action Aid
Meeting
with
unicef
Meeting
with
unicef
Nowshehroferoze HANDS
Arshad Ali Shaikh 0301 220 7173 GhotkiTakhleeq
Foundation
Meeting
with
Meeting
with
Khair Bano 0334 262 3656 Kashmore HANDS
Meeting
with
unicef
Meeting
with
unicef
Syed Jamil-ur-
Rehman
(0333) 260 1517
ja mi l_ra 2000@ya hoo.com
Yasir Majeed(0345) 518-8627
yas i r.ma jeed@a pexconsul ting.biz
APEX
Team
Based in Hyderabad with frequent visits to field and client meetings
Based in Islamabad with frequent visits for meetings and field visits
0332 308 4527Shanila Kiran
Team C
Team D
Team B
Team A
APEX Consulting Pakistan
UNICEF - Field Monitros & Reporters
Week 3 - Oct. 25 to Oct 30Week 2 - Oct. 18 to Oct 23Week 1 - Oct. 11 to Oct 16
Workplan & Team Deployment Matrix - upto October 31, 2010
Humanitarian Performance Monitoring Toolkit and Guide 01June2011
75
Weekly Field Monitor Summary Report
SUMMARY OF FINDINGS - Field Monitors Name of Organization:
Guide - 1 Location visit per day UNCEF Supplies PCA Report Verification Discussion with Key Informants and Affected Populations Other Issues
Date of
Visit
Name of Individual
Field MonitorsDistrict Sub-District Village/Camp Name of Site GPS Summary of status and observations Summary of accuracy of project reports
Issues around utility of project approach, exclusion, safety,
role of local government.Any urgent issues, unmet needs etc
1
2
3
4
5
6
7
Location (Adjust to local conext)
Humanitarian Performance Monitoring Toolkit and Guide 01June2011
76
Definitions
Focus Group Discussions
A focus group is a form of qualitative research in which a group of people are asked about
their perceptions, opinions, beliefs and attitudes towards a product, service or concept.
Questions are asked in an interactive group setting where participants are free to talk with
other group members
Participatory rural appraisal (PRA)37
Participatory rural appraisal (PRA) aims to incorporate the knowledge and opinions of rural
people in the planning and management of development projects and programmes
Hundreds of participatory techniques and tools have been described in a variety of books and newsletters, or taught at training courses around the world. These techniques can be divided into four categories:
• Group dynamics, e.g. learning contracts, role reversals, feedback sessions • Sampling, e.g. transect walks, wealth ranking, social mapping • Interviewing, e.g. focus group discussions, semi-structured interviews, triangulation • Visualization e.g. Venn diagrams, matrix scoring, timelines
To ensure that people are not excluded from participation, these techniques avoid writing wherever possible, relying instead on the tools of oral communication like pictures, symbols, physical objects and group memory. Efforts are made in many projects, however, to build a bridge to formal literacy; for example by teaching people how to sign their names or recognize their signatures.
37
See PRA Method and Application
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
77
Contracting Field Monitors – Template for SSA with Organization
UNITED NATIONS CHILDREN'S FUND
Terms of Reference for Institutional Consultancy “Field Monitor and Reporters”
1. Programme information:
Programme (No. & Name) :
Project (No. & Name) :
Sub-project (No. & Name) :
2. Purpose of assignment (Attach background documents, if necessary):
• Brief details of Humanitarian Situation
• Brief details of UNICEF interventions in humanitarian situation
• Brief detail on UNICEF humanitarian monitoring
• Overview of Role and requirements of Field Monitors: objective, purpose, timeframe,
numbers etc.
3. Duty Station:
4. Supervisor:
UNICEF Position / Unit /Section / Location
5. Major tasks to be accomplished:
Following is an example that can be adapted to suit the country / humanitarian context
a. Provide a team of Field monitors able to conduct monitoring and reporting based of
UNICEF interventions with a Coordinator to manage the day-to-day operations.
b. Ensure regular monitoring of UNICEF interventions in operational areas (in close
consulations and guidance from UNICEF) with timely, weekly reports on:
i) Delivery of UNICEF supplies to end-users (i.e. tracking of UNICEF supplies from
UNICEF warehouse to the end user, identifying any delays, bottle necks and
undelivered supplies)
ii) Specific issues/difficulties with regard to the UNICEF programme response (WASH,
Health, Nutrition, Education, Child Protection) including the effectivness of the
approach from the view of the end-user.
iii) The progress and performance of UNICEF cooperating and implementing partners
iv) The unmet needs of women and children in the affected areas
v) Key emergeing issues arising from Field monitoring, any social issues affecting
children or young, girls and women in general and any other related issues in the field,
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
78
inluding outbreaks, malutrition etc.
vi) Any special assignments required by the management in regards to quick
assessments, in-depth analysis of specific case/s
vii) Regular meetings between Field Monitors and UNICEF programme staff for direct
feedback / discussion. (Frequency to be determined with UNICEF provincial office)
6. End Products
Following is an example that can be adapted to suit the country / humanitarian context
Weekly summary report in the agreed format including:
a. Supply distribution and delivery progress (end-use information with identified gaps)
b. Progress of program response and intervention
c. Emerging issues
d. Progress and performance ofUNICEF cooperating / implementing partners
e. Original Field monitor reports for reference
7. Time Frame/Duration:
8. Qualification/Special Knowledge/experience: (to be worked out)
• Registered organization, preferably assessed by the UN with its own office and
qualified staff and able to undertake fulltime assignment at the specified provincial
project areas
• At least five years experiences with capacity to carry out field monitoring and
reporting of development/humanitarian interventions with reference to similar
projects
• Trained and qualified female and male (50% each) staff available for monitoring
and reporting (young energetic people with graduation from university/institute,
good endurance for rough conditions, and good communication in local ethnic
languages)
• Capacity to arrange all the logsitical arrangements for field staff, training and actual
field work
• Willingness to undertake the assingment in the given time period
9. Estimated Costs:
Esimated budget for 40 Field Monitors + 5 coordinators for 30 weeks
Province
(adapt as
necessary)
Number of
Field Monitors
(Jan-Jul 2011)
Average cost per
person
Estimated Cost for X
months
USD
Total
NB. Above figure includes all logsitics costs and US 20/week/Field Monitor cellphone airtime
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
79
10. Payment Schedule:
• 30% upon signing of contract and initial hiring and training of staff
• 40% end of the fourth month, subject to receipt of satisfactory timely, weekly
summary reports with accompanying background documents (original Field
monitor reports) and satisfactory organizational performance appraisal carried out
by provincial PME and verified by Chief Field Office.
• 30% end of the contract period, subject to receipt of satisfactory timely, weekly
summary reports with accompanying background documents (original Field
monitor reports) and satisfactory organizational performance appraisal carried out
by provincial PME and verified by Chief Field Office.
11 Prepared by:
Name Signature:_____________________
PME Officer Date:
12. Cleared by:
Name Signature:_____________________
Supply Officer Date:
13.. Approved by:
Name Signature:_____________________
UNICEF Representative Date:
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
80
Contracting Field Monitors – Template for Individual staff SSA
UNICEF
Terms of Reference for Individual SSA
Position Title: Field Monitor
Location:
Duration:
Reports to: Planning, Monitoring and Evaluation Section
Justification
Brief details of Humanitarian Situation
Brief details of UNICEF interventions in humanitarian situation
UNICEF is committed to effective transparent decision-making processes that are evidence-based
and facilitate timely and predictable humanitarian response. Furthermore, UNICEF is a results-
based organization. The broader humanitarian sector, largely through the IASC and the
humanitarian reform agenda, is committed to ensuring that the humanitarian sector continues to
strive to be more accountable in terms of results and impact in humanitarian response.
Brief detail on UNICEF country specific humanitarian monitoring
Overview of Role and requirements of Field Monitors: objective, purpose, timeframe, numbers etc.
Scope of work
As part of a UNICEF managed team of Field Monitors ensure regular monitoring of UNICEF
interventions in operational areas with a focus on:
i) Delivery of UNICEF supplies to end-users (i.e. tracking of UNICEF supplies from UNICEF warehouse
to the end user, identifying any delays, bottle necks and undelivered supplies)
ii) Specific issues/difficulties with regard to the UNICEF programme response (WASH, Health,
Nutrition, Education, Child Protection) including the effectivness of the approach from the view of the
end-user.
iii) The progress and performance of UNICEF cooperating and implementing partners
iv) The unmet needs of women and children in the affected areas
v) Key emergeing issues arising from Field monitoring, any social issues affecting children or young,
girls and women in general and any other related issues in the field, inluding outbreaks, malutrition
etc.
Note (delete as necessary)
Where feasible it is recommended that a 3rd
party organization is hired
to provide the management, coordination and hiring of Field Monitors
along with a Coordinator.
Directly hiring and managing 10-15 Field Monitors on SSAs has
hidden costs in time, transport and management and does not
address Access issues since they will be subject to the same UNICEF
movement restrictions
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
81
vi) Any special assignments required by the management in regards to quick assessments, in-depth
analysis of specific case/s
vii) Regular meetings between Field Monitors and UNICEF programme staff for direct feedback /
discussion. (Frequency to be determined with UNICEF provincial office)
Major duties, responsibilities and expected results
Major duties and responsibilities Expected results
80% • Field Visits to UNICEF operational areas to undertake monitoring activities
Record of performance of
UNICEF interventions
10% • Production of Field Visit reports
• Timely Field Visit Reports
5% • Debrief with PME and programme sections
• Direct feedback of Field Visits to UNICEF
5% • Ongoing orientation on UNICEF programme interventions
• Improved knowledge of UNICEF work
Desired background and experience
• At least five years of work experiences with capacity to carry out field monitoring and reporting of
development/humanitarian interventions with reference to similar projects
• Trained and qualified female and male (50% each) staff available for monitoring and reporting
(young energetic people with graduation from university/institute, good endurance for rough
conditions, and good communication in local ethnic languages)
Management
The post falls under the supervision/management of the Planning, Monitoring and Evaluation
Section (or similar CO arrangement).
The post will work in collaboration with the following, in particular:
• Programme Sections
• Supply and Logistics
• UNICEF PCA Partners
Submissions
Interested candidates should send their CV to _________________________
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
82
UNICEF Cluster / Sector Coordination Milestone Monitoring Tool
Q&A Guide on Using Tool
Contents of this section
• Why is this important (what the Country Management Team should know)
• What users should know
• What does it take to do it
• PCA Monitoring and Reporting Addendum Templates – tool (NB. Templates are pre-
completed with recommended priority high frequency indicators. CO should change as
necessary – refer to Indicator Guide)
Why is this important (what the Country Management Team should know)
• UNICEF accountabilities as country Cluster/Sector Lead Agency are an integral part of the
CCCs and the Humanitarian Reform agenda.
• The UNICEF Cluster/Sector Coordination Milestone Monitoring Tool is intended for quick use
as a Country Office management tool for high frequency monitoring of UNICEF’s progress in
meeting its cluster lead agency coordination accountabilities38.
o It is intended as a ‘minimum’ and provides a snapshot of UNICEF progress recognizing the
often limited CO monitoring capacity in the early stage of a response.
o It highlights the gaps that need to be addressed and can identify ‘quick wins’, i.e. ways to
strengthen clusters by sharing good practices across clusters and/or locations
• The Humanitarian PM tool is not intended for use by all cluster members and it does not seek
to assess the quality of UNICEF cluster coordination.
• Cluster performance monitoring tools that look at the performance of the entire cluster
response and quality of coordination which are often implemented with cluster partners, and
are more suited to lower frequency, in-depth monitoring, review or evaluation, have been
developed39 and links to these tools can be found on the Humanitarian PM Communities of
Practice filtered library40.
o Cluster performance monitoring tools are better suited for monitoring the Cluster / Sector
coordination in longer term chronic emergencies where clusters / sector working groups
have been long established.
• The Humanitarian PM tool attempts to distinguish Cluster Lead Agency responsibilities that
UNICEF can be expected to achieve alone, and those that require wider agreement and
cooperation with cluster / sector group members.
• UNICEF as country Cluster/Sector Lead Agency should be able to report on cluster coverage, but is not solely accountable for cluster/sector performance/coverage.
38
Derived from the Inter-Agency ToR for Country/Sector Cluster Lead 39
Tools developed through individual cluster initiatives and/or UNICEF Regional Office initiatives (e.g. draft CEE-CIS
cluster monitoring framework; current work on APSSC cluster monitoring framework) 40
http://intranet.unicef.org/emops/emopssite.nsf/root/PageCCCpm6
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
83
What users should know
• The tool can be applied in different ways: i.e. a self-assessment by the Cluster Coordinator; a
survey approach with one staff member carrying out one-to-one interviews with cluster
coordinators (ensuring some standardization in responses); or in a participatory assessment
by the cluster itself (suitable when the cluster is more mature).
What does it take to do it
• The key informant interview approach: in Pakistan, one-to-one telephone interviews were
carried out taking approximately 30minutes per interview
• Participatory assessment by the cluster may require a neutral, trusted facilitator.
See Communities of Practice Document Library for additional information on UNICEF role as
country Cluster/Sector Lead Agency
• Provider of Last Resort - definition
• ToR Generic Terms of Reference for Sector/Cluster Leads at the Country Level Operational
Guidance on cluster lead and OCHA Information Management
• Guidance Note on Determining Field-Level leadership of a Gender-Based Violence (GBV)
Area of Responsibility (AoR) Working Group in a cluster context
• Mental Health and Psychosocial Support in Humanitarian Emergencies
• Cluster Lead Agency joint letter on dual responsibility
• Draft Guidance on Cluster Co-Lead Chair arrangements
• Generic UNICEF Job Description for country Cluster Coordinator
Potential Challenges Solutions The cluster is not activated UNICEF accountabilities are the same for sector
working group coordination where the clusters are not activated
The government leads the clusters UNICEF has a stronger advocacy and supporting role to government to ensure accountabilities of cluster/sector coordination are met
Cluster members / donors are criticising UNICEF for cluster leadership
Assess UNICEF progress in reaching Cluster/sector coordination milestones Identify what the bottlenecks are and whether they are a sole UNICEF accountability or wider cluster/sector member accountability
* Please share Challenges and Solutions through the Humanitarian PM Community of
Practice for inclusion in future drafts of the Humanitarian PM guide.
http://intranet.unicef.org/CoP/EMOPSHPM/CommunityContent.nsf
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
84
Cluster / Sector Coordination Milestone Monitoring Tool (where UNICEF country cluster/sector – or sub-cluster- lead agency41)
Milestone Indicator MOV Responsibility Additional Guidance
Cluster / Sector Coordinator function is filled
Yes – Dedicated person Yes – Dual role / Interim Duel role42 No – Length of time function not filled
Arrival at duty station (Level/ Duration of Assignment)
UNICEF
Guide: "Filled" defined as • Dedicated person; • Person with dual role, • In some situations a dedicated person may not be appropriate or possible for cluster / sector working group coordination. • A decision on the need for a dedicated person (i.e. without a duel role) is best taken by the UNICEF CO in coordination with RO / HQ
Where UNICEF Co-lead. (Education, GBV) Have areas of responsibility been defined and agreed
Y (date) / N TOR / MOU UNICEF + Save UNICEF + UNFPA
Country example: Pakistan UNICEF and Save the Children See CoP Document repository
Partners are convened
List of cluster members on shared Inter-Agency (OCHA) platform
UNICEF
Agreement / ToR for Coordinating Mechanism is established
No – No action by UNICEF
TOR/MOU on shared Inter-Agency (OCHA) platform
UNICEF TORs should: • address key functions in the IASC Generic TOR for cluster coordinator http://OCHAonline.un.org/OCHALinkClick.aspx?link=OCHA)&docid=1085357 (ref IASC policy) • define roles and responsibilities including means of establishing committees or sub-groups to carry out different functions (ref good practice) • define clear mechanisms for overall coordination across sub-groups (ref good practice)
Partial – Drafting of TOR Cluster Coordinating Mechanism has been proposed/ is under development
Joint Cluster
Yes -- Cluster Coordinating Mechanism is finalized/agreed
Joint Cluster
Cluster Operational Strategy/ Action Plan is established
No – No action by UNICEF
Strategy or MOU on shared Inter-Agency (OCHA) platform
UNICEF A good strategy should: • present an analysis of context and corresponding programming priorities(ref good practice) • be revisited periodically based on analysis of progress/constraints and changes in context(ref IASC generic TOR) • be reflected in overall country strategy e.g. CHAP (ref IASC generic TOR) • include a exit or transition strategy for cluster(ref IASC generic TOR)
Partial – Drafting of Cluster Strategy has been proposed/ is under development
UNICEF
Yes -- Cluster Strategy is finalized/agreed Joint Cluster
41
Global Cluster Leads WASH, Nutrition, Education co-lead with Save the Children; Sub-cluster lead Child Protection, GBV co-lead with UNFPA 42
Specify if a Duel role is interim, i.e. until a dedicated person arrives
UNICEF Humanitarian Performance Monitoring - Toolkit and Guide 01 June 2011
85
Milestone Indicator MOV Responsibility Additional Guidance
Programme standards43
established and promoted
No – No action by UNICEF
Programme standards documents; partner organization reports on shared Inter-Agency (OCHA) platform
UNICEF IASC Cluster TOR: “It is important that cluster members are aware of relevant sector policy guidelines and standards as well as relevant government commitments in line with international law” Cluster programming should also integrate attention to cross-cutting issues: diversity, environment, gender, HIV/AIDS and human rights, Do no harm
Partial – Drafting/development of programme standards has been proposed/ is underway
UNICEF
Yes established – Programme standards are finalized/agreed
National govt or Joint Cluster
Yes promoted – Programme standards are being promoted within member organizations
Joint Cluster
IM capacity established
No – No action taken TOR, workplan or JD; arrival at duty station (Level/ Duration of Assignment)
UNICEF
Yes -- Accountability for cluster IM established within cluster staff or CLA office and position staffed IM role staffed
UNICEF
Basic IM systems mapping coverage established (3W)
No – no 3W sharing mechanism initiated
Status and date of 3W tool as last updated; circulation or accessibility of tool on shared Inter-Agency (OCHA) platform
UNICEF
Partial – system or templates for sharing 3W circulated or posted on cluster e-platform in last 2 weeks
UNICEF
Yes – information contributed and circulated at agreed frequency
Joint cluster
Performance management systems in place
No – no action taken UNICEF
Partial stage 1 – process of agreeing monitoring indicators has been initiated44
Cluster minutes on shared Inter-Agency (OCHA) platform
UNICEF
Partial stage 2 – monitoring indicators have been agreed AND reporting frequency
Cluster minutes on shared Inter-Agency (OCHA) platform
Joint cluster
Yes – performance monitoring data has been shared by 80% of cluster members at the agreed schedule
Performance monitoring reports or data available on shared Inter-Agency (OCHA) platform
Joint cluster
Action taken to "fill identified gaps" and to address duplication (yes/partial/no)
No – monitoring data is not available to identify gaps or has not been used to do so Yes - Actions taken to fill gaps: programme response, advocacy or fundraising
Cluster minutes on shared Inter-Agency (OCHA) platform
Joint cluster
43
It is recognized that while programme standards may have been agreed at the global level, at the national level there is likely to be the need for some adaption to the local context and for UNICEF national partners to agree and formalize their commitment 44
UNICEF as cluster member can propose agreed UNICEF priority sector indicators as a starting point for discussion on cluster performance indicators
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
86
Cross-Sectoral Communication for Development (C4D)
Q&A Guide on Using Tool
Contents
• Why is this important? (What the Country Management Team should know)
• What the User Should Know
• Draft C4D Indicators
• Annex: Relevant Extracts on C4D in UNICEF and Inter-Agency Policy
Why is this important? (What the Country Management Team should know) Communication for Development is critical for humanitarian action results from several different angles:
• Promoting lifesaving behaviour changes including: hygiene and safe water treatment; safe infant and young child feeding practices; use of bednets; response to mine/ unexploded ordinance risk and HIV/AIDS prevention, care and treatment.
• Accountability to affected populations on the scope and nature of the humanitarian action (which also contributes to Prevention of Sexual Exploitation and Abuse by humanitarian actors).
• Engagement of affected populations as rights holders in defining needs and priorities, and in implementing and reviewing/assessing humanitarian action.
Justification for the above is well-established in various Inter-Agency references including
• SPHERE standards which also draws on experience of the Humanitarian Accountability Partnership45
• IASC Generic TORs for Cluster Coordinators46 Given the complexity and frequent cross-sectoral nature of this kind of C4D activity as well as the need to use the same channels to access affected populations, there are huge benefits and efficiencies to be gained from Inter-Agency coordination. Similarly, performance monitoring of the above, should be coordinated at an IA level. What the User Should Know Performance monitoring of C4D initiatives will require data collection at a selection of different levels: At the level of inputs, indicators are very limited but can give a preliminary perspective on the estimated potential coverage to check early implementation. They do not merit setting up an extensive monitoring system. They allow estimated coverage based on the catchment population of mass communication channels (e.g. radio messages, billboards), organizations or institutions that are enlisted in C4D activities. Estimates must take into account known constraints on communication channels such as literacy rates and access to radio as well as patterns in terms of groups accessing services or institutions used. At the level of outputs, performance indicators will be very much shaped by the strategies adopted. They should provide rough but realistic estimates on potential population coverage differentiating different levels of exposure to messages and/or engagement in behaviour
45
http://www.sphereproject.org/ http://www.hapinternational.org/default.aspx 46
http://ochaonline.un.org/OchaLinkClick.aspx?link=ocha&docid=1085357
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
87
change, participation or ownership. The indicators allow estimates based on numbers of events and estimates for different types of approach including:
• Mass media channels based on tracking numbers of radio spots aired, billboards placed, posters distributed;
• Public or facility-based events; • Small group discussion processes facilitated by trained communicators and community-
level workers; • Community-led processes.
At the level of quality of implementation, qualitative data collection can be used, piggy-backing on systematic field visits or using dedicated resources. This can provide important information on:
• how well messages are delivered;
• how well messages are received and barriers in terms of understanding and attitudes/beliefs;
• who is/is not reached with messages or engaged in a participatory manner and why/why not.
At the level of outcomes, statistical surveys are required to explore actual change in knowledge, attitudes and practices.
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
88
Draft Indicators
Indicators MOV Considerations
Life-saving messages Accountability to Affected Populations Input indicators # of radio stations engaged/ transmission radius/ % access radio # partner organizations engaged and estimated programmes catchment population
• # partner organizations engaged/ # agreed points for providing information/ receiving feedback /estimated catchment population
• Partner agreements
Output indicators
• Estimated # / % of total population exposed through: o Radio based on # spots aired / # radio stations/ transmission radius x % access radio; o Public events based on # events x category of size of event47; o Group discussion events based on # of events x estimated average persons/event / targeting general, men-women, boys-girls / county o Discussions/teaching in schools based on # of schools visited or involved / county o Community-led processes based on # of communities engaged
• Estimated # / % of total population informed about humanitarian programmes48 through: o Radio based on # spots aired / # radio stations/ transmission radius x % access radio; o Programme outreach or delivery points based on # of sites with information mechanisms set up to agreed standards (information boards) and/or # of meetings/group discussions # of events providing information on x estimated # participants / county o Community-led processes based on # of communities engaged
• Estimated # / % of total population with access to feedback mechanisms49 o # of sites with information mechanisms set up to agreed standards
• Partner reporting through Ministries/ Clusters
• Requires agreement on formulation of indicators across partners, sectors/clusters • Providing information on response and feedback mechanisms require agreement on standards for both.
Quality of implementation
• % of observed communications activities where messages were delivered accurately; • % of focus groups where messages were understood and received positively (no concerns or contradictory attitudes/beliefs) • % of focus groups or sites where issues were identified about who was reached/not reached through C4D / which groups were excluded (age, gender, ethnicity, host/ displaced, disability, HIV/AIDS) and reasons50
Same as for Life-saving messages, as well as: • % of sites where there were concerns/complaints about programme delivery that were/were not raised; reasons and nature of concern
• Joint monitoring missions or Third-Party Monitoring
Requires standardization of core issues to explore and qualitative methods for reliability of data; requires a high enough coverage of sites with clear criteria for sampling to ensure confidence in data
Outcome indicators
• % of target population recalling key messages; e.g. key hygiene and safe infant feeding practices
• % of population demonstrating changed behaviour; e.g. households with area for washing hands and soap
• % of affected population with awareness of humanitarian action in their community
• % of affected population with complaints/concerns having accessed feedback mechanism
• % satisfactory response to feedback – for affected population; by agreed standards
• Statistical Survey
47 Even in industrialized countries, techniques for estimates on participation in large public events are highly inaccurate and often politicized, with estimates often differing by 100%. It can be useful to simply categorize events for a rough calculation: e.g. <100; 100-250; 250-500; 500-750; 750-1000 etc. 48 See Humanitarian Accountability Partnership quality checklist. Providing information should ideally cover: what agencies are providing support; scope of response programmes; criteria for targeting affected population; progress updates and how feedback can be provided/complaints lodged. http://www.hapinternational.org/standards.aspx 49 See Humanitarian Accountability Partnership quality checklist. A safe and accessible system: allows affected populations know how they can feedback; clarifies the scope of dealing with complaints; includes a way of checking that they understand their right to complain; includes a means of tracking and responding to complaints or feedback. 50 Reasons can be categorized before or after based on analysis of open response
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
89
Annex: Relevant Extracts on C4D in Policy
Core Commitments for Children in Humanitarian Action (CCCs)
1.7 Human rights-based approach to programming UNICEF, with the support of its partners, is
committed to reinforcing a human rights-based approach to programming in humanitarian actions
by:...• Promoting the participation of children, adolescents, women and affected populations,
including in the analysis, design and monitoring of humanitarian programmes.
1.15 Communication for development Communication for development in emergencies seeks to
share relevant, action-oriented information so that when disaster strikes, people in affected
communities know what actions to take to maintain and protect the health and wellbeing of all
their members, including those with disabilities, the elderly and other especially vulnerable
groups. It is a consultative process among programme and communication specialists, local
authorities, change agents and communities. It is misguided to presume that communities
affected by humanitarian situations are too shocked and helpless to take on responsibilities. In
fact, many people, including children, are able to return to normalcy more quickly when they
participate in helping others and themselves during and after an emergency. Communication for
development can help to:
• Support programme sectors in consulting adults, children and young people in affected groups
from the onset of a crisis. This has been shown to be a key factor in reducing deaths and
promoting psychological healing, cohesion and social mobilization.
• Forge alliances and bring stakeholders together.
• Establish a central health education and communication coordination centre.
• Focus on establishing or re-establishing positive individual and social practices.
• Conduct a rapid assessment of communication channels and resources.
• Participate in sectoral assessments that help to identify high-risk practices that have implications
for behaviour change communication, as well as opportunities for developing community-based
response mechanisms.
Nutrition Commitment 6: Children and women access relevant information about nutrition
programme activities. Benchmark: Communication activities providing information on nutrition
services (including how and where to access them) and entitlements are conducted in all
emergency-affected areas
Health Commitment 4: Women and children access behaviour-change communication
interventions to improve health-care and feeding practices. Benchmark: All affected populations
are exposed to key health education/promotion messages through multiple channels.
WASH Commitment 4: Children and women receive critical WASH-related information to prevent
child illness, especially diarrhoea. Benchmark: Hygiene education and information pertaining to
safe and hygienic child-care and feeding practices are provided to 70% of women and child
caregivers.
Child Protection Commitment 8: The use of landmines and other indiscriminate or illicit weapons
by state and non-state actors is prevented, and their impact is addressed. Benchmark : Children
and communities in affected areas have access to mine/ unexploded ordinance risk education
and are better protected from the effects of landmines and other indiscriminate and/or illicit
weapons.
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
90
Education Commitment 5: Adolescents, young children and caregivers access appropriate life
skills programmes and information about the emergency, and those who have missed out on
schooling, especially adolescents, receive information on educational options. Benchmark:
Relevant education programmes are implemented, including for adolescents and young children.
HIV/AIDS Commitment 1: Children, young people and women have access to information
regarding prevention, care and treatment. Benchmark: 90% of affected population is reached and
provided with information on prevention, care and treatment.
IASC Country Cluster Coordination Terms of Reference
Sector/cluster leads at the country level are accountable to the Humanitarian Coordinator for
facilitating a process at the sectoral level aimed at ensuring the following:… Participatory and
community-based approaches • Ensure utilization of participatory and community based
approaches in sectoral needs assessment, analysis, planning, monitoring and response.
SPHERE
Common standard 1: participation The disaster-affected population actively participates in the
assessment, design, implementation, monitoring and evaluation of the assistance programme.
Note also that the forthcoming revision of SPHERE will include a newly framed common standard of “People-centred humanitarian response” which includes participation, community mobilisation, psychosocial, complaints and redress, information sharing)
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
91
Outcome Surveys
Contents of this section
• Why is this important (what the Country Management Team should know)
• What users should know
• What does it take to do it? Why is this important (What the Country Management Team should know)? • Where funding allows periodic population based surveys should take place, ideally as an
Inter-Cluster/Sector exercise.
• Data collection for surveys will be lower frequency (e.g. at 6 or 12 months) and higher cost
• UNICEF monitoring and programme sections should seek opportunities with planned
Cluster/Sector surveys to inform wider Humanitarian performance monitoring.
What the user should know? • Outcome (survey) indicators are included in the Indicator Guide (Part C) and can be
included in the Results Framework
• Planned surveys should feature in the Simplified IMEP
What does it take to do it? The following gives some example of Survey types
CO monitoring sections are best placed to advise on the best survey method in coordination
with relevant clusters/sectors
a) Nutritional SMART Surveys – http://www.smartindicators.org/
Purpose: The SMART Methodology Version 1 provides a basic, integrated method for
assessing nutritional status and mortality rate in emergency situations.
Use: It provides the basis for understanding the magnitude and severity of a humanitarian
crisis.
Content: for Reference see SMART Methodology
http://www.smartindicators.org/SMART_Methodology_08-07-2006.pdf
An optional food security component provides additional context for nutrition and mortality
data analysis.
b) Multi-Cluster Rapid Assessment Methodology (McRAM) – http://www.mcram.org/
Purpose: It provides a mix of quantitative and qualitative cross-sectoral situational data with-
in a short timeframe (where preparedness measures are in place that includes training in
use of Personal Digital Assistant (PDA) technology). It provides situational data using a
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
92
purposive sampling of sites. It was originally developed in Pakistan adapting from the IA
Initial Rapid Assessment tools, simplifying and adapting these to the context. It can be used
for this form of IRA at the beginning of a rapid onset emergency and in repeat exercises
where the situation changes significantly. It is generally organized as a coordinated inter-
Cluster exercise.
Content: Questionnaire, PDAs and pre-trained field research teams. These can be adapted
to cover a range of issues in the situation of the affected population.
Use: The questionnaire is designed to be used with community groups (separate male and
female groups at each site). A site may be specific, i.e. a village or camp; or an area where
people have been displaced to.
It can be used to assess the needs of affected populations displaced to host communities
and camps (formal and non-formal).
Adapting to Country Context: The decision to utilize McRAM should ideally take place at the
Inter-Agency level as part of preparedness planning.
c) Multiple Indicator Cluster Survey (MICS) http://www.childinfo.org/mics.html
Survey tools
The MICS survey tools are developed by UNICEF after consultations with relevant experts
from various UN organizations as well as with interagency monitoring groups. UNICEF
works closely with other household survey programmes, in particular the Demographic and
Health Surveys (DHS) programme, to harmonize survey questions and modules and to
ensure a coordinated approach to survey implementation, with the objective to provide
comparability across surveys and to avoid duplication of efforts. The survey questionnaires
are modular tools that can be adapted to the needs of the country.
Implementation and capacity building
MICS surveys are typically carried out by government organizations, with the support and
assistance of UNICEF and other partners. Technical assistance and training for the surveys
is provided through a series of regional workshops where experts from developing countries
are trained on various aspects of MICS (questionnaire content, sampling and survey
implementation, data processing, data quality and data analysis, and report writing and
dissemination).
Survey results
Results from MICS, including national reports and micro level datasets, are widely
disseminated after completion of the surveys and can be downloaded from the MICS pages
on childinfo.org.
Increasing periodicity in MICS4
Starting with MICS4, UNICEF now provides assistance to countries at more frequent
intervals - every three years instead of every five years. This provides the opportunity for
countries to capture rapid changes in key indicators, particularly the MDGs.
Humanitarian Performance Monitoring Toolkit and Guide 01 June 2011
93
Potential Challenges Solutions
•
•
•
* Please share Challenges and Solutions through the Humanitarian PM Community of
Practice for inclusion in future drafts of the Humanitarian PM guide.
http://intranet.unicef.org/CoP/EMOPSHPM/CommunityContent.nsf