revision of the monitoring and evaluation framework -...

1

Revision of the Monitoring

and Evaluation Framework

10th Meeting of the Global Steering Committee (GSC) Video conference, 12 March 2015

2

Contents

1. Background ............................................................................................................................ 4

2. Justification ............................................................................................................................ 5

3. The main changes to the revised framework ........................................................................ 6

3.1 The principles guiding the revision of the framework ..................................................... 6

3.2 The major changes included in the framework ............................................................... 6

4. The revised Monitoring and Evaluation plan ......................................................................... 7

4.1. Levels of information ...................................................................................................... 7

Table 1: Elements of monitoring and dimension of study at each level of information ... 7

4.2 Monitoring roles and responsibilities .............................................................................. 8

4.3 Monitoring tools .............................................................................................................. 9

4.4 Monitoring products ...................................................................................................... 10

4.5 Timing of monitoring ..................................................................................................... 11

Table 2: Calendar of monitoring activities ....................................................................... 11

4.6 Monitoring resources ..................................................................................................... 11

4.6.1 Human resources .................................................................................................... 11

Table 3: Human resources for M&E ................................................................................. 12

4.6.2 Financial resources ................................................................................................. 12

Table 4: Financial Resources for M&E ............................................................................. 12

5. Revised consolidated logical framework ............................................................................. 13

5.1. Impact indicators and Outcome indicators .................................................................. 13

Table 5: GS Impact and Outcome Indicators ................................................................... 14

5.2 Output indicators ........................................................................................................... 14

5.2.1 Consolidated GS framework at output level .......................................................... 14

Table 6: Consolidated logical framework ........................................................................ 16

5.2.2 Consolidated GS framework at activity level .......................................................... 18

Table 7: Activity report template ..................................................................................... 19

6. Evaluation ............................................................................................................................ 21

Table 8: Midterm evaluation workplan ........................................................................... 22

3

List of Annexes:

Annex 1 – Consultancy team proposal for a revised M&E framework

Annex 2 – Reactions to the proposal of the consultancy team

Annex 3 – Revised Narrative Monitoring Templates

Annex 4 – Proposal for revising the M&E framework – 7th GSC meeting

4

1. Background

Effective monitoring and evaluation, together with good planning, play a major role in enhancing the effectiveness of the implementation of the Global Strategy (GS). Through monitoring, stakeholders receive regular feedback on the progress made towards achieving the goals and objectives of the programme; pursuant to evaluation, a rigorous and independent assessment of completed or ongoing activities, drawing upon data generated from monitoring, should determine the extent to which objectives are achieved.

The monitoring and evaluation framework of the GS is structured around the programme logical framework, the core element for assessing the implementation of the programme. The Global Office and each Regional Office have prepared logical frameworks that fit into the global logical framework, to enable annual reporting of the programme’s outputs as a whole.

The programme’s expenses are reported though specific financial reporting templates, which present the project expenditures at the output level.

Although the bases for the former monitoring framework were appropriate, the monitoring tools in place did not allow for the full implementation of the monitoring activities. As a concrete example, the logical frameworks of the Regional and Global Offices were developed at different stages, and some included an extensive set of indicators that, in certain cases, could be better classified as activities. In addition, there was no clear definition of the monitoring products; therefore, the monitoring had been subsumed within the compilation of the annual narrative reports. In view of all these constraints, the 7th Global Steering Committee (GSC), held in October 2013, endorsed the proposal for revising the existing Monitoring and Evaluation (M&E) framework, noting the following three main areas of concern:

(a) The programme logical framework was not seen as providing sufficient detail with regard to the process of monitoring and evaluating, particularly on the following aspects: (i) how the global and regional frameworks were harmonized; (ii) when monitoring should take place; (iii) how monitoring had to be carried out; and (iv) what resources were required;

(b) The existing programme logical framework needed fine-tuning. It was felt that some of the existing indicators could be significantly improved and that others could be developed to take into account the need for monitoring at the more detailed level of activities; and

(c) The monitoring of expenditures required improvement. The existing framework for monitoring programme expenditures only went as far as the output level, and it was necessary to consider whether it should be extended to a more detailed level, in line with the proposed changes to the programme logical framework mentioned above.

To review the M&E framework, the Global Office hired two M&E experts, who assessed the existing framework and prepared the proposal for an Integrated Monitoring and Evaluation Framework (Annex 1). This proposal has been carefully assessed by the Global Office and the main recommendations have been included in the revised M&E framework. A detailed assessment of the recommendations of the consultancy team has also been included (Annex 2).

5

2. Justification

Following the GSC’s endorsement of the proposal for revising the M&E framework, and on the basis of the recommendations made by the consultancy team, the Global Office of the GS has revised the former GS M&E framework to strengthen the programme’s effectiveness at global, regional and country levels. The primary functions of the revised framework are:

1) To ensure that monitoring is a continuous management function that aims primarily to provide programme coordinators at global and regional levels with regular feedback and early indications of progress – or lack thereof – towards the achievement of intended results. In this connection, the revised framework aims to enhance participating partners’ capacities in evidence-based programming and monitoring, so that the results achieved may be better measured and corrective measures may be undertaken as appropriate.

2) To ensure that monitoring tracks actual performance, against what was planned or expected according to approved action plans and annual workplans. This generally entails the continuous collection and analysis of data on programme processes and results. In this respect, the revised framework includes a monitoring plan that clarifies the roles and responsibilities of each partner and the monitoring tools that they should use.

3) To ensure that monitoring is results-oriented, meaning that its main goal is to provide better reports to stakeholders on the progress made in terms of output and outcomes. To this end, the annual narrative reporting templates have been revised, and partners will also report at activity level through activity reports.

4) To clarify the steps in conducting monitoring activities, and ensure that the knowledge generated and lessons learnt by participating partners are better captured and disseminated. In this connection, the main monitoring tools have been identified and all the monitoring tools and deliverables are now included in the new calendar of monitoring and evaluation deliverables.

5) To offer a standard tool for all regions, including the underfunded regions that have yet to complete their regional action plans. In this connection, the consolidated logical framework has been developed (see Section 5 below).

6) The revised framework has been defined, under the assumption that all resources have been mobilized. This will allow the Global Office and Regional partners to adjust their year-end targets according to the resources actually received.

6

3. The main changes to the revised framework

3.1 The principles guiding the revision of the framework

In line with the former framework, the revised M&E framework is based on the Programme’s three pillars and its four outputs, as established in the Global Action Plan. On the basis of the issues mentioned above, the key principles guiding the revision of the M&E framework have been:

• To emphasize accountability by improving existing common standards and defining clear roles and responsibilities for partners;

• To distinguish levels of information. The Framework is built on the basis of the programme level, which serves as the starting point for the global, regional and country levels.

• To prioritize output indicators by distinguishing output indicators from activity indicators.

• To harmonize the GS framework with the frameworks of existing donors, to facilitate reporting against donors’ frameworks.

• To enhance coordination between partners. The revised framework was completed through a participatory approach, by involving Regional Offices and the GSC members.

• Ensure cost-effectiveness by maximizing the use of existing skills and resources.

3.2 The major changes included in the framework

Based on the key principles guiding the revision of the M&E framework, the issues noted and endorsed by the 7th GSC meeting and the recommendations made by the consultancy team, the major changes in the revised framework are:

1) The inclusion of reporting at country level. In this connection, the new framework is based on the following four levels of information: (i) programme (GS) level, (ii) Global Office level, (iii) regional level and (iv) country level.

2) A reduction in the number of output indicators. For this purpose, an assessment of all existing outputs was conducted. As a result, all the new indicators are specific, measurable, attainable, reliable, and time-bound (SMART).

3) The inclusion of reporting at activity level. The assessment exercise was followed by a categorization exercise through which the new indicators were grouped as output or activity indicators.

3) The harmonization of the existing frameworks to enable aggregation. To this end, the logical frameworks of the Global Office and the two implementing regions were overlaid, to assist the systematic identification of the areas in which harmonization across the various levels of the global programme could be achieved.

4) Strengthening the relevance of the Integrated Budget as a tool for improving the GS’s financial planning and monitoring. The Integrated Budget indicates the resource requirements and serves as the tool for planning the expenditures needed on an annual basis, according to the envisioned budget.

7

5) Expanding the reporting coverage by preparing six-month activity reports. These reports will provide members with an update on the activities implemented at global, regional and country levels on a semestral basis.

6) Definition of an annual Monitoring workplan, including respective roles and responsibilities.

4. The revised Monitoring and Evaluation plan

4.1. Levels of information

The GS was designed according to the following levels of information: programme level, Global Office level, regional level and country level. In this connection, the results achieved at Global Office, regional and country levels will be aggregated to build the results at the programme level. To this end, the programme level will ultimately provide a complete and coherent view of the GS’s overall progress and impact. In this respect, the various levels of analysis include:

Programme level: GS level (Catalytic and Implementation programmes): This is the starting point for the M&E process. The information, conclusions and findings at global, regional and country levels will be consolidated at programme level.

Global level: assessment of the activities implemented by the Global Office.

Regional level: assessment of the interventions made by regional participating partners.

Country level: this level of analysis only applies to countries where technical assistance and training is being provided. The monitoring and reporting responsibilities at country level will be managed directly by the Regional Offices.

Table 1: Elements of monitoring and dimension of study at each level of information

Information level

Element of Monitoring

Dimensions of Study

Programme level

Progress towards the programme logframe (aggregation of Global Office, regional and country logframes)

Results achieved by the programme as a whole.

Progress towards the achievement of the impact, outcome and outputs of the programme.

Global level

Progress towards the Global Office logframe

Coordination of the programme and efficiency of the governance structure.

Progress towards the implementation of the research agenda.

Production of guidelines and training materials.

8

4.2 Monitoring roles and responsibilities

The M&E activities fall under the responsibilities of the Global Office and the Regional Offices.

1) Programme level: The Global Office manages the M&E responsibilities at programme level, as it is responsible for designing and coordinating the M&E plan and for ensuring that the Regional Offices adhere to the GS monitoring and reporting requirements and standards. Under this role, the Global Office is responsible for ensuring that the implementation is undertaken in line with the GSC’s decisions, providing guidance to the regional activities on how to implement the monitoring activities, conduct the mid-term and final evaluation and perform oversight activities. In addition, on an annual basis, the Global Office is responsible for collecting, aggregating and consolidating the progress and activity reports submitted by each regional partner into the annual consolidated narrative progress reports and activity reports. The consolidated reports ensure that the elements of the programme are clearly presented in the reports submitted to the GSC members. 2) Global level: At global level, the Global Office is responsible for monitoring the Global Office component and producing the Global Office annual and activity reports.

3) Regional level: At regional level, each Regional Office contributes to the implementation of the M&E plan by adapting its regional M&E to the regional templates. In line with the reporting requirements, each Regional Office should consolidate the progress achieved by all regional partners, including the reporting at country level. Furthermore, the regions are responsible for providing the baseline information and monitoring the established benchmarks.

4) Country level: The monitoring responsibilities of activities implemented at country level remain at the level of the regional implementing partners, as they are directly responsible for interventions within the target countries. The activities implemented at country level can

Information level

Element of Monitoring

Dimensions of Study

Regional level

Progress towards the Regional Office logframe

Regional interventions of the programme.

Technical assistance provided to countries in the region.

Capacity-building activities, including workshops conducted in the region.

Country level

(reported by the

regions)

Reported through the annual consolidated report and the regional progress reports

Added value of the GS within target countries.

Contribution of the programme towards the development of the SPARS.

Countries’ capacities to produce agreed minimum set of core data.

Strengthening of national statistical capacities.

9

be monitored through technical assistance missions, participation of target countries to regional workshops and/or seminars, etc.

4.3 Monitoring tools

As noted above, the M&E Framework is a pyramidal strategy in which the primary source of data consists of the monitoring and evaluation reports at each level. To ensure that the information is based on reliable data, the following monitoring tools will be used:

1. Monitoring Output Indicators: to measure progress and trends in the short- and medium-term. As mentioned above, the indicators of the Global Office and the Regional Offices have been harmonized to enable the aggregation of their results at programme level. In addition to these output indicators, Regional Offices and the Global Office may establish their own internal indicators that will allow them to better assess the implementation of the programme at their level.

2. Monitoring Activity Indicators: besides the regular monitoring of the activities indicated in the workplan, each Regional Office and the Global Office will prepare six-month progress reports indicating their progress towards the activity indicators identified by the revised framework.

3. Field Visits: these are an essential part of the monitoring, mostly taking the form of technical assistance missions conducted by the Regional Offices. In addition, some field visits will be organized to prepare and produce the mid-term and final evaluations.

4. Evaluations: these are performed to assess the relevance of the programme at each level of implementation (programme, global, regional country). The mid-term and final evaluations will focus on the design, process, results and impacts. The Global Office is responsible for leading the evaluations, in close cooperation with the Regional Offices and input from the GSC.

5. Desk Reviews and Data Collection and Analysis: the data for these activities are to be drawn from a variety of sources, particularly from the Country Assessment questionnaires and reports, including the light Country Assessment questionnaire to be distributed and assessed by the Regional Offices.

6. Integrated budget: the integrated budget is the financial tool that enables the assessment of the Global Office and Regional Offices’ annual workplans and budgets, against the budget approved by the GSC members.

7. Participation in Governance and Coordination meetings: GSC meetings, Regional Steering Committee (RSC) meetings and coordination meetings are an opportunity to discuss progress, identify problems and agree on possible solutions and corrective measures, as appropriate.

10

4.4 Monitoring products

The Global Office and each Regional office are expected to follow regular monitoring processes, strengthen self-assessments of progress, improve documentation of the implementation processes and facilitate timely modifications as required. In line with the new monitoring requirements, the reporting templates have been updated (Annex 3). In addition, the frequency of reporting has been extended to allow for the monitoring at activity level (Annex 3). In this respect, the frequency of the monitoring activities will be the following:

- Annual narrative progress reports: Each participating partner will prepare its annual narrative and financial progress report following the standard reporting templates developed by the Global Office. These reports have been prepared on the basis of the programme’s four outputs, to enable aggregation of results and inform members on the progress of the programme as a whole.

The annual narrative reports will be first consolidated at the regional level by each Regional Office. This regional report will be submitted to the Global Office before 31 March of the following year. Upon this, the Global Office will incorporate the information from the regional reports, including the Global Office report, into the annual consolidated narrative report. This report will be submitted to the GSC members by 30 April of the following year.

- Financial reports: these will be prepared following the standard financial reporting templates of the GS, to present and certify the expenditures incurred by each participating partner by output. The annual financial reports are submitted to the Fund Administrator for aggregation before 30 April of the following year. Upon submission, the reports are aggregated and submitted to the GSC members by 31 May of the following year.

- Midyear financial report: in addition to the annual financial report, each participating partner prepares and certifies its midyear financial report and submits it to the Fund Administrator no later than 30 September of the same year. The Fund Administrator will then aggregate the financial reports and submit them to the GSC members by 30 October of the same year.

- Activity report: the new monitoring framework incorporates the six-month progress reports within the activity reports. Each Regional Office and the Global Office will report on the progress on their activities on a six-month basis, covering the periods of January – June and July – December. The regional activity reports and the Global Office activity report will be aggregated by the Global Office and circulated to the GSC members no later than 15 September of the same year and before 28 February of the following year.

11

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

l

l

l

l l

Reports prepared by implementing partners (Regions and GO)

l Reports consol idated by GO and sent to GSC

6 months consolidated activity report

Annual partners financial reports

Annual Consolidated financial report

Narrative

reports

Activi ty

reports

Financial

reports

Annual consolidated narrative report

Consolidated midyear financial report

6 months partners financial report

Annual partners narrative reports

6 months partners activity reports

4.5 Timing of monitoring

In the former M&E framework, most monitoring activities were subsumed within the annual narrative and biannual financial reports. In this connection, the revised framework calls for more periodic reviews to strengthen self-assessments of progress, improve documentation of the implementation processes and facilitate timely modifications, as needed. As a result, each participating partner will use the monitoring tools listed in Section 4.3 above and the monitoring products to be submitted by each partner (Section 4.4 above). The timing of monitoring activities will be as follows:

Table 2: Calendar of monitoring activities

4.6 Monitoring resources

4.6.1 Human resources

For a comprehensive view of the resources devoted to M&E, it is necessary to examine the structure of the functioning Regional Offices and of the Global Office. Currently, resources have been mobilized for implementing the activities in Africa, Asia and the Pacific and at the global level. In this connection, human resources can only be analysed in the regions that are implementing the GS.

As indicated in Table 4 below, the regional office in Africa is the only office that has a full-time position devoted to performing M&E activities, while the Regional Office in Asia and the Pacific and the Global Office have decided to include M&E activities within the regular tasks of their staff members. Table 4 below assesses the capacities at the existing levels of implementation and each participating partner’s contribution towards the outputs of the programme.

12

Global Office Regional Africa Regional A&P Regional LAC Regional NE Regional CIS Total

Development M&E framework 130,000 130,000

Implementation Monitoring framework 230,000 273,000 200,000 200,000 100,000 100,000 1,103,000

GS Evaluation 280,000 280,000

Total M&E budget 640,000 273,000 200,000 200,000 100,000 100,000 1,513,000

total budget by region 20,353,604 25,996,839 13,808,944 13,878,494 4,927,499 4,865,974 83,831,353

% of M&E against the total budget 3% 1% 1% 1% 2% 2% 2%

Budget- Monitoring and evaluation

Table 3: Human resources for M&E

4.6.2 Financial resources

In line with the integrated budget approved by the GSC members, M&E activities appear as a specific funded activity under each regional workplan. The total cost of these activities amounts to 2% of the programme’s total integrated budget.

As indicated in Table 5 below, most of the resources have been allocated at global level, to fund the development of the revised framework and the programme’s midterm and final evaluation. The allocation of resources at regional level is consistent with the number of target countries at regional level.

Table 4: Financial Resources for M&E1

1 Values as indicated in the revised Integrated budget, submitted during the 10th GSC meeting.

Participating partner Outputs Existing M&E mechanisms

and capacities of partner

Full time M&E position in the

Regional Office

Regional Office in Africa -

AfDB

1, 2, 4 Harmonized Logframe

Yes. A M&E consultant is working for this project on long term basis.

UNECA 1, 2, 4 Harmonized Logframe

No. The Programme Officer will perform the M&E activities.

Regional Office in Asia

and the Pacific - FAO RAP

1, 2, 4 Harmonized Logframe

No. The M&E activities will be partly performed by the Regional Coordinator, in consultation with the Regional Office staff.

ESCAP 1, 2, 4 Harmonized Logframe

No. This component will be monitored by the Regional Office in Asia and the Pacific.

Global Office 1, 2, 3, 4 Harmonized Logframe No. The Programme Coordinator and the Programme Officer will perform the M&E activities.

13

5. Revised consolidated logical framework

As explained above, the revised monitoring framework was prepared following a wide consultation with participating partners, and in cooperation with two M&E experts. As a result of the revision exercise, the global and regional frameworks have been harmonized on the basis of the GS’s three pillars and four global outputs. This harmonization exercise has substantially assisted by identifying measurable output indicators, including the country level as a new dimension for monitoring, and establishing key activity indicators at all levels.

The revised monitoring framework provides more commonality in the approaches taken by the regions and the Global Office towards meeting the global outputs and, eventually, the outcomes and impact, while keeping the same level of flexibility in implementation that was incorporated in the beginning of the GS.

The revised monitoring framework is presented as a series of tables, starting with the impact and outcome of the programme and then with the four global outputs and the implications in terms of Global Office, regional and country indicators. The regional indicators will be adopted by all regions, including those that have not yet developed their regional action plan. This does not imply that regions cannot have their separate individual logframes, which may measure any additional key element that may be of interest to them. However, these individual logical frameworks will not be aggregated as they have not been designed under the commonality approach.

The output indicators will be followed by a presentation of the new activity indicators.

These play an intermediary role, as they allow regions to monitor progress in the short term

while also helping to maintain the number of output indicators at a reasonable level.

5.1. Impact indicators and Outcome indicators

The Impact indicator for the GS is monitored using the average score on the use of statistics in the policy-making process, a methodology designed by PARIS21. While the PARIS21 scoring system is the only existing measure for the use of statistics in policymaking, it is affected by serious drawbacks: the methodology involves a degree of subjectivity, and only analyses the intended use of statistics in policy documents; also, the basis for computing the indicator is of only marginal relevance to agricultural and rural statistics. In view of the limitation of this score system, the Global Office will work on the development of a new statistical capacity indicator, which is more tailored to the agricultural sector and more focused on the effective use of statistics in policymaking.

As noted by the consultancy team, there is an overreliance on Country Assessments when generating the Means of Verification (MoV) for most of the indicators, especially because Country Assessments are conducted every two years. The Regional Coordination Units are developing a lighter, more focused survey questionnaire to be distributed on an annual basis to generate the information required on a faster and more frequent basis.

The Outcome indicator directly reflects the GS’s three pillars. The MoV of this output consists of the Country Assessments to be performed by each Regional Office. The Country Assessment monitors the improvements in target countries’ “agricultural statistical capacity” and relies on complex computations, based on 23 different capacity dimensions

14

Indicators Means of verification Baseline Target Risks and mitigations

IMP

AC

T

Improved evidence-based decision

making for poverty reduction,

increased food security,

sustainable agriculture and rural

development

Average score on the use of s tatis tics in

the pol icy-making process *

PARIS21 Scoring System

on the Use of Statis tics52,9 70

Risks:

1. Lack of (or weak) demand for evidence-based pol icy-making.

2. Lack of national support and interest in providing s tatis tics for

agricul tura l and rura l development.

Mitigation measures:

1. Closely involve users ' group in planning and producing s tatis tica l

outputs , including in the defini tion of the core set.

2. Increase advocacy, Implement effective communication in

disseminating s tatis tica l outputs .

Number of countries producing agreed

minimum set of core data of adequate

qual i ty (Pi l lar 1)

Country assessments

and progress reports

Number of target countries that have

integrated agricul tura l and rura l

s tatis tics into their national s tatis tica l

systems (Pi l lar 2)

Country assessments


Number of target countries that have

improved national coordination

mechanisms and s tatis tica l legis lation

to foster the susta inabi l i ty of s tatis tics

(Pi l lar 3)

Country assessments


Number of target countries with improved

agricultura l s tatis tica l capacity (Pi l lar 3)

Country assessments


* A new indicator wi l l be proposed by the end of 2015

OU

TC

OM

E

Target countries are enabled to

develop sustainable statistical

systems for the production and

dissemination of accurate and

timely agricultural and rural

statistics, comparable over time

and across countries.

Risks

1. Lack of national pol i tica l interest in improving agricul tura l and rura l

s tatis tics .

2. Ineffective insti tutional coordination at country level .

3. Global s trategy activi ties may not respond to country priori ties .


1. Commitment of the target countries to support implementation of the

GS is one of the selection cri teria .

2. Country assessment proposals and process to identi fy country

priori ties .

100% of

funded

countries

0

involving institutional infrastructure, resources, statistical methods and practices, and the availability of statistical information, most of which are derived from Country Assessment questionnaires.

Table 5: GS Impact and Outcome Indicators

5.2 Output indicators

The new set of indicators is the result of a careful assessment of the existing output

indicators at programme, Global Office and regional levels. The assessment exercise began

by prioritizing the existing indicators and distinguishing between the outputs that contribute

directly to the programme, and other intermediate outputs that contribute to the Global

Office and regional activities and therefore could be understood as activity outputs.

Considering that most of the activities under Outputs 2 and 4 occur at country level, it was

acknowledged that the role and importance of target countries may not have received due

attention. Thus, the regional indicators were assessed to better measure the effectiveness

of the implementation at country level. Through the consultative process, regions have re-

estimated their baseline data. In addition, benchmarks have been established for those

regions in which the GS is being implemented (Africa and Asia and the Pacific).

5.2.1 Consolidated GS framework at output level

Considering the above observations, the consolidated logical framework has been developed around the four programme outputs and the four levels of reporting (programme level, Global Office, regional level and country level).

15

According to the information flow included into the consolidated logframe, the reporting coverage of Regional Offices also includes the information from target countries. In this respect, the information provided by the Regional Offices will then be aggregated, by the Global Office, with the information reported at global level. Through this aggregation, the indicators at regional level (which also captures the information at country level) and Global Office level will be aggregated in their correspondent programme indicators, which summarize, into one single value, implementation efficiency across all levels of intervention.

As an example, Output Indicator 1.4, “Number of annual workplans endorsed by the governance of the programme”, is the result of the aggregation of the Global Office output indicator “Number of annual Global Office workplans endorsed by the GSC” and the Regional indicator “Number of annual regional workplans endorsed by the GEB [Global Executive Board]”. Since the workplans endorsed by the governance of the programme are only at global and regional levels, this indicator does not include any information at country level. The table consolidates the logical framework, including the baseline information, the targets, the MoVs and the risks and mitigations. In addition, the table also shows the corresponding output indicators at global, regional and country levels to facilitate the aggregation procedure.

The consolidated logical framework is structured under the four outputs of the programme, namely:

- Programme Output 1: “Effective governing bodies set up and functioning at global and regional levels” concerns the internal structure and functioning of the GS, more concretely its governance; administrative and operational requirements and thus outputs and indicators are restricted to regional and Global Office levels.

- Programme Output 2: “Coordinating bodies of the national statistical system, legal frameworks and strategic plans established (by the countries) in the target countries, to enable the integration of agriculture into the national statistical system”. This output is fully implemented at regional and country levels; thus, the progress towards the achievement of this output indicator is reported directly at regional level.

- Programme Output 3: “New cost-effective methods for data collection, analysis and dissemination developed and disseminated”. This output is fully implemented by the Global Office; therefore, there are only Global Office indicators. The results will be made available to the national statistical systems through the adaptation, technical assistance and training provided by the Regional Offices as described in Output 4 below.

- Programme Output 4: “Increased capacity of agricultural statistics staff in regional training centers (i.e. trainers) and target countries”. This output is implemented at global and regional levels. The global indicator will measure the production of guidelines and training materials that will be used and adapted by the Regional Offices to provide technical assistance and training at regional level. Therefore, the global indicator will measure the availability of tools for providing technical assistance and training, while the regional indicators will measure the effectiveness of these interventions at country level. In view of all these elements, the consolidated logical framework reported below was designed, as a unique tool for integrated reporting at all levels of intervention.

16

Global Office level Regional level Country level Programme level

Programme indicator Baseline Target MoV Global Office indicators Regional Level indicators Country level indicators Risk and mitigations

Number of governance

meetings organized at

global and regional level

243 meetings (6 GSC + 12 GEB + 18 RSC

+ 7 coordination meetings) Minutes of meetings

Number of governance and

overa l l coordination meetings

organized at global level

Number of governance meetings

organized at regional level -

Number of Regional

action plans developed1 5 Regional Plans - Number of Regional action plans

completed -Number of annual

consol idated narrative

reports and annual and

midyear financia l

reports submitted

0

12 annual consol idated progress

report and 5 midyear financia l

reports

Annual reports

Number of annual consol idated

narrative and financia l reports

submitted -Number of annual

workplans endorsed by

the governance of the

programme

0 (6 Africa , 6 A&P, 6 LAC, 6 RNE, 6 CIS) Annual workplans Number of annual global office

workplans endorsed by the GSC

Number of annual regional

workplans endorsed by the GEB -Number of target

countries that have

improved national

coordination

mechanisms and

s tatis tica l legis lation

090 (40 in Africa; 20 in As ia ; 20 in LAC;

5 CIS; 5 NE )

Regional and consol idated

progress reports -Number of target countries which

have improved national

coordination mechanisms and

s tatis tica l legis lation.

Number of target

countries that have

integrated agricul tura l

and rura l s tatis tics into

their NSDSs .

090 (40 in Africa; 20 in As ia ; 20 in LAC; 5

CIS; 5 NE )

Regional and consol idated

progress reports -Number of target countries have

integrated agricul tura l and rura l

s tatis tics into their national

s tatis tica l s trategies .

Number of target

countries where

additional government

funding is provided to

support agricul tura l

s tatis tics .


5 CIS; 5 NE ) CA questionnaire -

Number of target countries where

additional government funding is

provided to support agricul tura l

s tatis tics .

Risks:

1. Lack of national pol i tica l

interests in setting up the

insti tutional and

organisational s tructures .

2. Non-functional national

coordination s tructures .


1. Commitment of the

countries to support the

implementation.

2.inclus ion of the

coordinating s tructures in

the development plans .

3. Advocacy campaign.

Ou

tpu

t 2

Ou

tpu

t 1

Programme level

Risks:

1. Global and Regional

governing bodies are not

effective

2. Lack of cons is tency

between Regional Action

Plans and Global Action

Plan.


1. Faci l i tate the decis ion

making process for the

governing bodies .

2. Global Office supports the

development of the regional

action plans

Table 6: Consolidated logical framework

17

Number of technica l

reports , guidel ines and

tra ining materia ls

developed and

disseminated

0 80 progress report

Number of technica l reports ,

guidel ines and tra ining materia ls

developed and disseminated - -Number of target

countries that have

adopted a minimum of 5

cost effective methods


5 CIS; 5 NE ) CA -

Number of target countries that

have adopted a minimum of 5


-Number of countries

producing agreed

minimum set of core

data of adequate qual i ty


5 CIS; 5 NE ) progress report -

Number of countries producing

agreed minimum set of core data

of adequate qual i ty

Number of target

countries who have

received appropriate

tra ining on the use of



5 CIS; 5 NE ) progress report -

Number of target countries who

have received appropriate

tra ining on the use of cost

effective methods

Number of research

topics completed by the

Global Office

ou

tpu

t 3

Ou

tpu

t 4

Risks:

1. Technica l di fficul ties at

country level to apply new

methods .

2. Countries and tra inees

are not selected on the

bas is of needs .

3. High turnover of tra ined

s taff.


1.More concerted efforts in

adopting new

methodologies .

2. Ensure appropriate

selection and deployment

of tra inees .

3. Request recipients of

tra ining grant to s ign an

agreement for serving in the

government for a minimum

number of years

Risks:

1. Research does not

respond to the priori ty

needs of the regions .

2. delays in fina l i s ing

technica l guidel ines


1. Consultation with

countries to ensure

relevance of research efforts

2. Bui ld s trategic

partnership with

specia l i sed insti tutions on

research activi ties

--Number of research topics

completed by the Global Office

Technica l reports and

minutes of SAC meeting 300

18

5.2.2 Consolidated GS framework at activity level

In connection with the justification for a revised M&E framework, one of the major concerns expressed by the GSC members was the lack of reporting at activity level. In addition, the consultancy team suggested that the number of existing output indicators could be better classified as activities, to take into account the need for monitoring at the more detailed level of activities.

In view of these issues, a categorization exercise of the existing indicators was conducted to distinguish output and activity indicators. This categorization exercise was followed by a redefinition and prioritization of the indicators, to include new effective indicators at output and activity levels. As a result of this exercise, a set of activity indicators have been produced and included into the activity report.

As for the consolidated logical framework, the activity report is constructed around the four outputs of the programme, to facilitate the identification and monitoring of results. This also allows for the results at programme level to be aggregated, as the activity indicators directly contribute to the achievements of the output indicators.

For understandable reasons, the number of activity indicators is substantially higher than the number of output indicators. In this connection, the activities will be reported at Global Office, regional and country levels. The reporting at country level will be provided by the region, to distinguish the regional activities that have an impact at regional level from those that have a direct impact at country level.

Each region will prepare the activity report outlined below on a six-month basis, to present the main activities implementation by the GS at all levels during the first and second semesters of each year. As explained in the reporting section, these reports will be aggregated and circulated to the GSC members by the Global Office.

As indicated in Table 8 below, the activity report is very light. This avoids overwhelming both partners and GSC members, as the completion of these reports is not excessively time-consuming, but at the same time provides GSC members with key summarized information regarding the implementation of GS activities at all levels.

19

Table 7: Activity report template

Programme level Global Office level Regional level Country level

Programme indicator Global Office indicators Regional Level indicators Country level indicators

Number of governance

meetings organized at

global and regional

level

Number of GSC meetings ,

GEB meetings , coordination

meetings organized

Number of RSC REB meetings

(or a l ternative governance

group) organized -Number of annual

Consol idated financia l ,

narrative and activi ty

reports submitted

Annual narrative and

financia l reports submitted

to the GSC

Annual regional narrative

and financia l regional

reports submitted to the

Global Office-

-Number of miss ions to target


identi fication of core data -

-Number of target countries

that have improved national

coordination mechanisms


that have improved


- Number of workshops on

SPARs organised -- Number of miss ions in


SPARS-

- Number of target countries

which have completed the

SPARS

- Number of advocacy

workshops organized -


where additional

government funding i s

provided to support

agricul tura l s tatis tics

Number of research topics

with a methodologica l bas is

prepared

Number of expert meetings

organized to discuss the

research topics

Number of research topics

field tested.

Ou

tpu

t 1

Number of target

countries that have

integrated agricul tura l

and rura l s tatis tics into

their NSDSs .

Number of target

countries where

additional government

funding i s provided to

support agricul tura l

s tatis tics .

Ou

tpu

t 2

-

Ou

tpu

t 3

Number of research

topics completed by the

Global Office

Number of target

countries that have

improved national

coordination

mechanisms and


-

20

Number of guidel ines

and tra ining materia ls

developed and

disseminated

Number of guidel ines and

tra ining materia ls

developed and

disseminated

Number of syl labus , curricula

and teaching materia ls

produced -

-Number of backstopping

miss ions per country on the

adoption of New

methodologies & data

harmonization

-- Number of workshops on

cost effective methods -


that have adopted a

minimum of 5 main cost

effective methods

Number of countries

producing agreed

minimum set of core

data of adequate

qual i ty

-Number of countries

producing agreed minimum

set of core data of adequate

qual i ty

-Number of miss ions to

support tra ining

development -- Number of scholarships

provided -- Number of tra ining

workshops organized at

regional and country level-


who have received

appropriate tra ining on the

use of cost effective

methods

Ou

tpu

t 4

Number of target

countries who have

received appropriate

tra ining on the use of


Number of target

countries that have

adopted a minimum of

5 main cost effective

methods

21

6. Evaluation

The mid-term evaluation will assess the performance of the Global Strategy in achieving its desired results according to workplans also in consideration of the global environment in which the programme operates. It will highlight the likelihood for sustainability of the results achieved as well as any issues and challenges affecting efficiency and effectiveness of the programme, and recommend corrective measures, if applicable.

In connection with the Global Action plan and donor agreements, the mid-term evaluation of the GS will start in April 2015 and will be finalized by November 2015 over approximately 26 weeks.

It will pave the way for improved project delivery for the remaining the duration of the project (end of 2017) and propose amendments (if any) required in GS design, implementation arrangements and/or institutional linkages in order to effectively and sustainably contribute to the achievement of the goals of the programme. More concretely, the objectives of the evaluation are to:

(i) assess the performance (effectiveness, efficiency and relevance) of the Global Strategy and provide guidance on how to improve its results;

(ii) assess the extent to which the Global Strategy remains relevant to the agricultural statistical capacity development agenda; and

(iii) assess the impact of the current funding gap on the programme including on its sustainability, and formulate recommendations for programme’s way forward

The midterm evaluation will be conducted by an independent consultancy team, under the overall management of FAO’s Evaluation Office, which has an independent status.

The GSC will endorse the ToRs of the midterm evaluation as well as will receive the midterm evaluation report, while the GEB will endorse the selection of the consultancy team and will provide comments on the draft evaluation report. The Global Office will support the Evaluation Office by providing the necessary background and documentation. Under its secretariat role, the Global Office will submit the documentation to the GEB members for comments and will prepare a detailed assessment of the draft evaluation report, for submission to the GEB and GSC as applicable.

It is expected that the midterm evaluation will be implemented as outlined in Table 8 on the following page.

22

Tasks Respons ibi l i ties 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

Supervision Overall supervision of the process OED

Identification of evaluation team , finalization TorS, contracts OED

Review of documentation, consultations, preparation of evaluation tools Evaluation Team

Comments on the evaluation tools and matrix Global Executive Board

Evaluation matrix and tools and finalized Consultants

Visit to Global Office Evaluation Team

Visit to Regional Office in Africa and field visits in Africa Evaluation Team

Visit to the Regional Office in Asia and the Pacific and Field Visits Evaluation Team

Visit to donors and other stakeholders Evaluation Team

Preparation of draft evaluation report Evaluation Team

Review of draft report Global Executive Board

Revision of the draft report Evaluation Team

Submission of final report Global Steering Committee

Inception

phase

Field Visists

Evaluation

report

Timel ine (weeks)

Table 8: Midterm evaluation workplan

revision of the monitoring and evaluation framework -...

Documents