implement the programme ‘promoting birth registration … · implement the programme ‘promoting...

plan-international.org Case study title | Date

In April 2012, Plan International Ireland received funding from the Human Dignity Foundation (HDF) to implement the programme ‘Promoting Birth Registration for Children’s Development and Protection’ in Liberia, Sierra Leone and at the international level. The aim of the Programme was to contribute to the promotion of social and economic growth in Sierra Leone and Liberia and to enhance the protection of children’s rights, through promoting birth registration in country and internationally. After two years of successful implementation, HDF granted Plan International Ireland an additional year of funding with the intent to finalise some of the initiatives commenced, strengthen ownership and programmatic sustainability and focus on resource mobilisation in the long run. Unfortunately, in May 2014 the Ebola Virus Disease (EVD) hit both countries of intervention with devastating effects on every aspect of social life. The Programme activities were put on hold for over a year as Plan, implementing partners and relevant institutions reacted to the health and humanitarian emergency.

As the situation improved, a post-Ebola assessment was carried out in both countries starting in August 2015. This assessment determined the effects of Ebola on the birth registration Programme and informed revisions to the Programme strategy and activities going forward. Learning from these findings and from a field visit conducted in September 2015, a programme revision took place. The overall Programme strategy, activities, targeted geographical areas and profile of beneficiaries generally remain the same. However, some adjustments were made to the Programme activities, targets and implementation methodologies to ensure that they were appropriate considering the current post-Ebola context. HDF approved these amendments and activities re commenced in January 2016.

Plan Ireland is now recruiting an evaluation consultant/ team of consultants to complete an evaluation of the programme from April 2012 to November 2016

Main title – Secondary title 2

Plan is an international child centred development organisation working in 50 developing countries. With a focus on long term sustainable development, Plan assists children and their communities across a range of sectors including education, health, livelihoods and water/sanitation. Established in 2003, Plan Ireland supports Plan’s work worldwide with a particular focus on West Africa. Funding is received from a variety of sources including through the public via child sponsorship, Irish Aid, EU, trusts, foundations, trade unions and corporate firms.

Plan Ireland and HDF have been working in partnership to enhance the protection of child rights in Liberia and Sierra Leone through the promotion of birth registration. The Programme mobilised demand at the community level, contributed to improving the registration systems locally and nationally (including by discussing the introduction of an ICT solution for birth registration), and advocated for the introduction of policies and frameworks that favour immediate and cost-free registration. The combination of these initiatives allowed for the registration of a total of 445,188 children in the first phase of the programme.

The current programme identifies 3 Specific Objectives:

1. To increase registration rates of children under 13 in Programme targeted areas of Liberia, and children under 5 (Western Area District) and under 18 (Port Loko and Kailahun Districts) in Sierra Leone. 2. To contribute to a births and deaths registration system that is digitalised, child rights-based and integrated into governmental plans for the establishment of a nation-wide CRVS system. 3. To increase awareness and knowledge on birth and civil registration among selected governments and within the Plan family.

The Plan country offices in Liberia and Sierra Leone implement the projects in partnership with relevant local ministries and local implementing partners manage activities on the ground. In particular the following partners are actively involved in the project:

Sierra Leone: National Office for Births and Deaths (NOBD) at the national level and the District Management Heath Teams (DHMT) of Western Area, Port Loko and Kailahun Districts at the local level.

Liberia: the Department of Vital Statistics at the national level and Community Safety Initiative (CSI), Youth for Environmental Services Inc. (YES), Centre for Justice and Peace Studies (CJPS), Defence for Children International (DCI) and Liberian Environmental Care Organisation (LECO) at the local level. Concerning objective 3 and the international focus of the programme, Plan International Ireland has also worked with the Plan International Headquarters to build support furthering the global discussion on Birth Registration/CRVS including in collaboration with selected governments in Asia and the Pacific by building knowledge base and tools on BR/CRVS.

The overall purpose of the evaluation is to assess the overall project impact of the programme as set out in the projects results framework and in relation to the whole programme period (April 2012-November 2016). The evaluation will contribute to shared learning and provide accountability to


partners, beneficiaries and donors. It will identify, through a review of the project implementation strategy, lessons learned and make recommendations to guide and inform future similar projects and programs.

The evaluation will assess the outcomes and contribution to the impact of the project against key parameters including the project’s relevance, effectiveness, efficiency, sustainability, timelines of activity implementation, and its strengths and weaknesses. In addition, the strengths, weaknesses and challenges during the programme period implementation will be analysed with a view to extract lessons and best practices for the future.

The final evaluation will provide the decision-makers in HDF, Plan, other donors and the wider public with sufficient information to:

a) make an overall independent assessment about the impact of the project paying particularly

attention to the project outcomes;

b) Identify key lessons and to propose practical recommendations for follow-up actions.

The evaluation will be based on the five evaluation criteria endorsed by the OECD-DAC (relevance, efficiency, effectiveness, sustainability and impact). The evaluation will also assess the integration and impact of cross cutting issues in the project.

The key parameters to be assessed by the evaluation are:

a) Relevance: describes how well the project has addressed the problems of the beneficiaries

and how well it matches the development policies and strategic objectives of the

country/region.

b) Efficiency: stands for how well the inputs are transformed into output and outcomes. Value for

money should also be assessed

c) Effectiveness: measures the degree to which the project’s outputs have provided benefits and

contributed to the project purpose.

d) Impact: describes how and to what extent the project has contributed to the solution of the

problem and to the achievement of the overall objective.

e) Sustainability: measures the likelihood of a continuation in the stream of benefits produced by

the project after the period of external support has ended. To what extent can the programme

outcomes be sustained beyond the lifespan of the programme

f) Mutual reinforcement (coherence): assesses the extent to which activities undertaken

complement partner country's policies and other donor interventions. The consultants are

required to use their professional judgment and experience to review all relevant factors and

to bring these to the attention of the project partners.

The evaluation will cover the accomplishment of all the expected results as outlined in the project results framework during the whole programme period (April2012-November 2016). All the project result areas, outcomes and activities stated in the logical framework matrix of the project will be assessed (these documents are available on request to interested parties).


Liberia County

Bomi

Bong

Gbarpoli

Grand Bassa

Grand Cape Mt

Lofa

Margibi

Montserrado

Nimba

Sierra Leone Kailahun

Port Loko

Western Area

The consultant will select sample districts and communities from the project region in consultation with relevant stakeholders and under specific objective 3, in order to assess progress made in increasing awareness and knowledge of the birth and civil registration system with other governments and within the wider Plan International network, remote interviews with relevant key stakeholders participating in the programme will be conducted.

The consultant will design an impact evaluation methodology that assesses the programme achievement. Innovative methodologies are encouraged.

The specific tasks of the evaluator will be to:

a) Develop an evaluation framework and methodology and refine these with the project team.

b) Develop a sampling frame that will be discussed and adopted by the consultant

c) Develop an evaluation plan

d) Undertake extensive document review of the project

e) Conduct field visits to collect primary information as defined in methodology design

f) Prepare the evaluation draft report and present the findings to Plan International Ireland and the

respective Country Offices as well as to the donor (HDF)

g) Prepare and submit the final report to Plan International Ireland and the respective Country

Offices. The report will be eventually shared with the donor and published on HDF/Plan website.

The evaluation approach should be developed and implemented in three main phases: An inception phase, a field investigation phase and a synthesis and feedback phase.

I. Inception Phase This phase will involve the thorough and systematic review of the relevant programming documents related to the project, as well as documents shaping the wider strategy/policy framework. The consultant will analyse the logical framework in order to determine relevant issues and evaluation questions. Specific tasks to be undertaken during the inception phase will include:


Elaborate the indicative methodology.

Interview the project management team and key focal persons.

Present each identified evaluation question stating the information already gathered and

provide a first partial answer to the question, identify the issues still to be covered and the

assumptions still to be tested, and describe a full method to answer the question.

Identify and present the proposed list of tools to be used and applied in the field/data

collection phase for review and feedback;

Prepare and submit a detailed work plan with an indicative list of people to be interviewed,

surveys to be undertaken, dates of visit, itinerary, and name of team members involved.

Define issues and gaps requiring further analysis;

Prepare and share inception report of not more than 6 pages

II. Field Investigation Phase The Field Phase should start upon approval of the inception Report by Plan’s evaluation contact point. The consultant will:

Undertake the field mission applying the plan developed during the Inception Phase. This plan

has to be applied in a way that is flexible enough to accommodate any last-minute challenges

in the field. If any significant deviation from the agreed work plan or schedule is perceived as

creating a risk to the quality of the evaluation, these should be immediately discussed with the

evaluation manager.

Hold a briefing meeting with the project teams in the first days of the field phase.

Ensure adequate contact and consultation with the key project staff

Ensure adequate contact and consultation with, and involvement of, the different stakeholders;

working closely with the relevant government authorities and agencies during their entire

assignment. Use the most reliable and appropriate sources of information and harmonise data

from different sources to allow ready interpretation.

Ensure remote contact and consultations with Plan International Headquarters team and

relevant stakeholders for the international component related to the 3rd Objective.

Summarise field work at the end of the field phase, discuss the reliability and coverage of data

collection, and present preliminary findings in a meeting with the project management team

and relevant government administrators.

All data sets and original completed questionnaires and data collection sheets will become the property of Plan International Ireland and will be shared with the draft Evaluation Report

III. Synthesis and Feedback Phase This phase is mainly devoted to the preparation of the draft final report. The consultant will make sure that:

Their assessments are objective and balanced, affirmations accurate and verifiable, and

recommendations realistic.

When drafting the report, they will acknowledge clearly where changes in the desired direction

are known to be already taking place, in order to avoid misleading readers and causing

unnecessary irritation or offence.


The consultant will submit the draft evaluation report to the Plan International Ireland evaluation contact point for review. On the basis of comments expressed by Plan and HDF (the donor), the consultant will amend and revise the draft report. On the basis of comments made by participants, the consultant will prepare the final version of the report and will present this to Plan Ireland and to the Country Offices through an in country workshop.

The following are the qualification requirements for the evaluation consultant:

Extensive experience in the evaluation of technical assistance projects, preferably, those

under the child rights portfolio;

Demonstrable experience in impact evaluation

Fully conversant with the principles and working methods of project cycle management.

Knowledge of the activities of multilateral development donors will be an added advantage;

Solid knowledge of, and practical experience with gender analysis and planning;

Full working knowledge of English and excellent report writing skills

Competence and adequate experience in the use of qualitative and quantitative methods of

data collection and analysis including: sampling, desegregation of data, structured and semi-

structured interviewing, focus group discussions, and observation and triangulation research

methods.

Ability to interpret and analyse complex qualitative and quantitative data, and to present

findings and recommendations in a clear and concise way.

Excellent inter-personal communication skills including experience of facilitation and

presentation.

Ability to work equally well with communities and international organisations.

Knowledge and sensitivity to political and social contexts of West African Communities.

Experience in West Africa (particularly Liberia and Sierra Leone) would be an advantage.

The evaluation will be carried out from December 12 to 24 February 2017

The consultant will be required to submit a technical proposal indicating the number of days and rate for the consultancy work with a realistic action plan. Indicative dates for phases of the programme of works outlined below.

Phase Envisioned Timeline

Inception Phase 12-19 December

Field investigation Phase and International component

09- 20 January

Draft report 03 February

Final report 24 February


The overall supervision of the evaluators will rest with Plan Ireland. The consultant will report to a designated manager. The consultant will be expected to work closely with, and interact with the relevant in country staff and implementing partners.

Logistics: The consultant’s travel from base to the field and back will be included in the current

contract

Budget: the maximum budget available for this assignment is USD 40,000. The bid analysis team

will identify the better quality standards offered at best overall value for money.

Professional fee: The consultant will propose the professional fee as part of their financial

proposal, detailed to be inclusive of international travel costs, visas, and accommodation in the

two countries as well as other expenses related to visit to IH (London) or other expenses related to

international component (conference call, interviews).

Tax and insurance: The consultants shall be responsible for their income tax and/or insurance

during the assignment.

Code of conduct: The consultant is bound by the principles and conditions of Plan’s code of

conduct and child protection policy.

A contract will be signed by the consultant/evaluator upon commencement of the evaluation which

will detail additional terms and conditions of service, aspects on inputs and deliverables.

All expressions of interest should include:

Cover letter: A short (maximum two pages) letter addressing the evaluation criteria.

A detailed curriculum vitae of key evaluation staff

At least 2 references of organizations for whom you have carried out similar work

A sample of work completed by the consultant on a similar assignment

Technical Proposal (maximum five pages) interpreting the understanding of the TOR, detailed

methodology of executing the task, as well as draft evaluation framework and plan.

An approximate timeline for the work (based on timeline above)

The financial proposal should :

o Provide cost estimates for services rendered including daily consultancy fees related

to the consultant and/or associate consultant who will take part in the final evaluation

of the project.

o Include all expenses (transport cost; insurance, per diems, stationeries, and supplies)

related to all the different phases of the assignment.

o Include all expenses related to Dissemination of reports to Plan and stakeholders.

A PDF scanned copy of the tender cover letter signed by an authorized officer of the

consultancy team outlining the contents of the tender documents

The above information should be forwarded to [email protected] before 5pm GMT on Friday, November 25th 2016. E-mails containing tender documents should have the subject: “Tender for the birth registration final evaluation”. Tenders and other e-mails sent to this address will not be open until after this date.


Alternatively tender documents can also be posted to Plan Ireland. Tender documents submitted by post should be in a sealed envelope and should be addressed as follows;

To the Birth Registration Final Evaluation Tender Committee

Plan International Ireland

126 Lower Baggot Street

Dublin 2

Ireland

If you have any queries about the tender process or would like to request a pack of related project information please e-mail [email protected] or call Plan International Ireland on +353 6599601 before the closing date.

The contract shall be awarded on the basis of expertise and experience as well as cost and adherence to necessary time frame. The team must be available to begin the assignment immediately and complete it by February 24th, 2017.


The following key aspects of the evaluation must be addressed.

1. Project Preparation and Design

How appropriate was the project design?

What preparatory activities were carried out?

Were they relevant?

Was there any outstanding omission in the design?

Were they reflected in the final project design?

Were any necessary preparatory activities omitted?

Was the initial intervention plan/logic appropriate?

Recommendations for improving future project design.

2. Relevance:

The main focus will be on the appropriateness of the project’s concept and design to the improvement of access to birth registration in each country. In particular the consultant must assess:

How the project has addressed access to birth registration for beneficiaries.

Extent to which the stated objectives addressed the problems of the primary target groups

Relevance of the project design within the framework of the country plans

Relevance of the project design within the framework of Plans child rights strategy

Relevance of the project in aligning with national strategies.

How the programme ensured accountability towards beneficiaries?

Level of significance of the results to address the primary needs of the beneficiaries.

Level of awareness of the children, implementing partners and other relevant stakeholders on

the results of the project.

3. Efficiency:

The main focus will be on how well the project activities transferred the available resources into the intended results in terms of quantity, quality, and timeliness through sustainable and participatory processes. In particular:

How optimal were project inputs (funds, expertise, time, and other resources etc.) converted

into results in the required quantity and quality and in good time?

Was the use of the project resources cost-effective? Was the project value for money?

Could the project activities have been done more efficiently?

Whether similar results or more could have been achieved at a lower cost in the same amount

of time?

Whether project activities were done right i.e. on time, in expected quantity and quality, and

through participatory processes.

Partner country contributions from local institutions and government (e.g. offices, experts,

reports, materials, labour), target beneficiaries and other local parties.

Responsiveness of project management to changes in the environment in which the project

operates;

Co-operation among project partners and other key stakeholders in achieving project results.

4. Effectiveness:

The main focus will be on the extent to which the project achieved its stated results and purpose. Key parameters to be assessed include:


The progress made in achieving the results of the project at all levels. The project results

Framework will be used as the basis for analysis and assessment; whether the intervention

logic actually worked or not.

Efforts made in capacity building of the partners and other local project stakeholders and

whether the strategies worked or not, and why.

Project management of risks taking into account the stated risks and assumptions.

Adequacy of management arrangements as well as monitoring and evaluation, technical

backstopping support given to the project by all parties concerned; coordination and

communication.

The level of local ownership of the project and its commitment to support the project after the

expiry of the project period.

5. Sustainability:

The sustainability criteria relates to whether the positive outcomes of the project and the flow of benefits are likely to continue after external funding ends or non-funding support interventions (such as: policy dialogue, coordination).

The final evaluation will make an assessment of the prospects for the sustainability of benefits on basis of the following issues:

the ownership of objectives and achievements, e.g. how far all stakeholders were consulted

on the objectives from the outset, and whether they agreed with them and continue to remain

in agreement;

policy support and the responsibility of the beneficiary institutions, e.g. how far donor policy

and national policy are corresponding, the potential effects of any policy changes; how far the

relevant national, sectoral and budgetary policies and priorities are affecting the project

positively or adversely; and the level of support from governmental, public, business and civil

society organizations.

institutional capacity, e.g. of the Government (e.g. through policy and budgetary support) and

counterpart institutions; the extent to which the project is embedded in local institutional

structures; if it involved creating a new institution, how far good relations with existing

institutions have been established; whether the institution appears likely to be capable of

continuing the flow of benefits after the project ends (is it well-led, with adequate and trained

staff, sufficient budget and equipment?); whether counterparts have been properly prepared

for taking over, technically, financially and managerially;

the adequacy of the project budget for its purpose particularly phasing out prospects;

socio-cultural factors, e.g. whether the project is in tune with local perceptions of needs and of

ways of producing and sharing benefits; whether it respects local power- structures, status

systems and beliefs, and if it sought to change any of those, how well-accepted are the

changes both by the target group and by others; how well it is based on an analysis of such

factors, including target group/ beneficiary participation in design and implementation; and the

quality of relations between the external project staff and local communities.

financial sustainability, e.g. whether the products or services being provided are affordable for

the intended beneficiaries and are likely to remained so after funding will end; whether enough

funds are available to cover all costs (including recurrent costs), and continued to do so after

funding will end; and economic sustainability, i.e. how well do the benefits (returns) compare

to those on similar undertakings once market distortions are eliminated.


technical (technology) issues, e.g. whether (i) the technology, knowledge, process or service

introduced or provided fits in with existing needs, culture, traditions, skills or knowledge; (ii)

alternative technologies are being considered, where possible; and (iii) the degree in which

the beneficiaries have been able to adapt to and maintain the technology acquired without

further assistance.

Wherever relevant, cross-cutting issues such as gender equality, environmental impact and

good governance; were appropriately accounted for and managed from the outset of the

project.

6. Impact:

The main focus will be on whether the project has contributed to the overall project goal, specific changes that the project has brought about in the lives of target groups, and impacts realized or likely to be realized in relation to cross cutting issues such as gender, environment and child protection. In particular;

Most significant changes that can be attributed to the project.

Achievements of the project against the original results (outcomes), outputs, and activities

specified in the project log frame.

Extent to which the project has impacted on digital birth registration

Positive/negative changes, intended/un-intended that the project produced on the target

groups both past and present

Also assess whether the impacts of the project:

o have been facilitated/constrained by external factors

o have produced any unintended or unexpected impacts, and if so how have these

affected the overall impact.

o have been facilitated/constrained by project/programme management, by co-

ordination arrangements, by the participation of relevant stakeholders

o have contributed to economic and social development

o have made a difference in terms of cross-cutting issues like gender equality,

environment, good governance, conflict prevention etc.


For each country the Report shall be completed in English and the relevant country official language following the format as follows:

1. Executive Summary

A succinct, to the point and free‐standing Executive Summary is an essential component. It should be no more than two or three pages. It should focus on the key purpose or issues of the evaluation, outline the main points of the analysis, and clearly indicate the main conclusions, lessons learned and specific recommendations.

2. Main body of the report

The main body of the report shall elaborate the points listed in the Summary. It will include references to the methodology used for the evaluation and the context of the action. In particular, for each key aspect (Relevance, efficiency, effectiveness, impact and sustainability) there should be a corresponding section according to the weighting agreed. Lessons learned (operational and developmental) should be incorporated. Recommendations should be as realistic, operationally sound and pragmatic as possible; that is, they should take careful account of the circumstances currently prevailing in the context of the action, and of the resources available to implement it locally.

3. Conclusions and Recommendations

4. Annexes

1) Terms of Reference

2) Project Performance Matrices

3) Data collection tools (Questionnaires, Focus Group Discussion guidelines etc.)

4) Itinerary for the Evaluation Team

5) List of Persons Consulted

6) Literature and Recommendation

7) Raw Data in digital form

implement the programme ‘promoting birth registration … · implement the programme ‘promoting...

Documents