care project standards measurement instrument

CARE International

CARE International Project Standards Measurement Instrument PSMI

Country Name of Project:

_______________________________________________

Person filling out form (and responsible for scoring):_______________________________________________ Other respondents (participants in the discussion): _______________________________________________ Date: ________________________________________________

Project status: a) proposal approval; b) self-evaluation; c) mid-term evaluation; d) final evaluation

Introduction to the CI Project Standards: 1 These CARE Project Standards apply to all CARE programming (including emergencies, rehabilitation and development) and all forms of interventions (direct service delivery, working with or through partners, and policy advocacy).

These standards, as well as accompanying guidelines, should be used to guide the work of project designers; as a checklist for approval of project proposals; as a tool for periodic project self-appraisal; and as a part of project evaluation. The emphasis should not be only on enforcement but also on the strengthening of capacity to be able to meet these standards for programme quality. At the time of initial approval, if a project can not meet one or more standards, allow for explanation of why, and what will be done about it. More than a passed/failed checklist, these call for a description of how well a project meets each standard.

Introduction to the CI Project Standards Measurement Instrument (PSMI):

To assist with measuring the degree of compliance with these standards (as called for in the introduction above), a series of more specific questions are proposed for each standard. These are intended to help the respondents (project staff using this PSMI to self-evaluate their own project, or senior managers to judge the compliance of a project with these standards) to consider the meaning of each standard, and to help in determining how well a project currently meets the standard. Though a rating scale is included for each standard, what may be more valuable is to document the evidence by citing or making reference to project documents (e.g. proposal, logframe, detailed implementation plan, M&E plan, etc.) and to comment on what the project is or will be doing to more adequately meet a particular standard.

The quantitative rating provides a measurement along a scale from does not meet at all to adequately to exemplifies a particular standard. These can help determine whether or not a project meets a standard well enough for approval at time of initial design. (Note: the minimum threshold levels will probably be different for immediate emergency projects, longer-term emergency and rehabilitation projects, and long-term development projects.) These scaled indicators can also help measure progress by a project towards more fully meeting each standard as it matures over time. Remember: more than just giving a numerical rating, it would be helpful to write a comment on why and in what way a project does or does not meet a particular standard.

The pattern used in this PSMI for each standard will be as follows:

a) The Project Standard b) The descriptive paragraph(s)

c) One or more leading questions, including requests to cite or make reference to where this topic is addressed within the project documents. Space (within the Word template) for entering the response.

d) A scaled rating giving an estimated quantification of how well this standard is currently met, based on the above considerations (circle, or, in the Word template, delete all except the chosen score). Rating score for Standard #x: 1 2 3 4 5

e) Request for comments on a) why the project currently may not fully meet this standard, and/or b) what is planned to more adequately meet it in the future. Comment:

The scaled ratings should be interpreted as follows:1 = This standard is not met, or at least it would be a stretch to say that this project even barely meets this standard at this time. L

2 = Probably meets this standard, under the circumstances, but fairly minimally. 3 = Meets this standard reasonably well. K

4 = We feel fairly confident that this standard is met quite well, at least as well as could be expected of most projects. 5 = Any objective observer should be able to agree that this project exemplifies this standard! J

Each CARE project2 should:1.be consistent with the CARE International Vision and Mission, Programming Principles and Values.Projects and programmes should fit comfortably within the spirit and content of the CARE International (CI) Vision and Mission statements. In other words, CARE projects should show how they will contribute, ultimately, towards lasting improvements in human well-being, hope, tolerance, social justice, reduction in poverty, and enhanced dignity and security of people. They should be guided by CI Programming Principles that synthesize and integrate with central elements of CAREs evolving programme approaches, including livelihoods, basic rights, gender and diversity, partnerships and civil society.

1. Can it be reasonably shown that this project contributes towards lasting improvements in human well-being, hope, tolerance, social justice, reduction in poverty, and/or enhanced dignity and security of people? Describe the evidence in the project proposal.

Response:

2. (In other words): Can it be demonstrated that the goal of this project contributes to one or more of the UN Millennium Goals? If so, which ones?

Response:

3. In consideration of the above questions, how consistent is this project with the CI vision, mission, values and programming principles? Rate on a scale of 1-5.

Rating score for Standard #1: 1 2 3 4 - 54. Please add a comment on why this project may not currently meet this standard as well as it should, and/or what will be done to more adequately meet the standard in the future.

Comment:

2. be clearly linked to a Country Office strategy and/or long term programme goals.Projects should not be isolated, but clearly embedded in long term multi-project programmes and strategic frameworks that address the underlying conditions and root causes of poverty and social injustice. Doing so provides a larger framework in which project decisions are made, but does not preclude strategic innovation and experimentation. CARE's strategies should be clearly linked to the development efforts of others (e.g. government, multilaterals, NGOs).

1. If there is a CO strategy (LRSP), how does the project plan shown how this project fits within and contributes to the priorities in that strategy? Response:

2. If there is a programme strategy covering the area or target population addressed by this project, is it clear how well this project fits within and contributes to that programme strategy? Cite or make reference to where this is addressed in the logframe or other project document.

Response:

3. Considering the above, how well do you think this standard has been met?


Comment:

3. ensure the active participation and influence of stakeholders in its analysis, design, implementation, monitoring and evaluation processes.Every project should be explicit about its process of participation and consultation, aiming for openness and transparency. Stakeholders will be understood to include target communities, partner organizations, governments, and CARE staff. The interventions of the various actors should be coordinated and reinforcing and, individually and together, work together to achieve sustainable impact.

1. How much were (or are) representatives of the target population involved in the design, implementation, monitoring and/or evaluation of this project? Describe that involvement and level of influence (or make reference to project documents that describe such participation).

Response:

2. How well do you think this standard has been met with regard to participation of the target population?

Rating score for Standard #3-a: 1 2 3 4 - 53. How much and in what ways were (or are) representatives of partner organizations or other stakeholders involved in the design, implementation, monitoring and/or evaluation of this project? Describe or cite references in project documents.

Response:

4. How well do you think this standard has been met with regard to partner organizations?

Rating score for Standard #3-b: 1 2 3 4 - 55. Please add a comment on why this project may not currently meet this standard as well as it should (with regard to target population and/or partner organizations), and/or what will be done to more adequately meet the standard in the future.

Comment:

4.have a design that is based on a holistic analysis of the needs and rights of the target population and the underlying causes of their conditions of poverty and social injustice. It should also examine the opportunities and risks inherent in the potential interventions. The diagnostic assessment and subsequent analysis should be based upon a clear frame of reference and include an analysis of problems and their causes from a range of perspectives including institutional as well as opportunity analysis. Social analyses could examine how needs and rights are related to gender, social class, ethnicity, religion, etc. The analysis should lead to an understanding of institutional capacity, power relationships, and the exercise of rights and responsibilities, as well as household level conditions.

1. Was the design of this project based on the findings of a diagnostic assessment of the problems and opportunities of the target population? If so, briefly describe the scope of that diagnostic assessment.

Response:

2. If so, how holistic was the diagnostic assessment? Did it include perspectives of more than one sector? Response:

3. How well did the diagnostic assessment look into underlying causes of the identified problem(s)? Cite references in the project documents that describe those underlying causes and how the project will address them.

Response:

4. Did the diagnostic assessment examine needs and rights related to gender, social class, ethnicity, religion, etc? If so, cite project documentation that describes the findings and recommendations of such analysis.

Response:

5. Did the diagnostic assessment include analyses of institutions that have responsibilities related to the target population, and the strengths and weaknesses of those institutions (as potential partners)? Cite references in the project documents.

Response:

6. Did the project design include examinations of the potential risks inherent in the choices of interventions? Cite references.

Response:



Comment:

5.use a logical framework3 that explains how the project will contribute to an ultimate impact upon the lives of members of a defined target population. The project plan should be clearly summarized in a logical framework that shows how proposed interventions and anticipated outputs will result in defined effects and impact. It should specify level of intervention (household, community, institutional, societal) and how the project will ultimately contribute to sustainable impact for a specific target population. It should identify key assumptions and provide validation for its central hypothesis.

1. Has the project design been summarized in the form of a logframe (logic model)? If so, attach a copy.

Response:

2. Does the logic model include goals, objectives, and indicators related to activities, outputs, effects and impact? (Shown on logic model.)

Response:

3. Is there a clear logical linkage and cause-effect relationship among these levels? (Examination of logic model.)

Response:

4. Is it clear whether the interventions address households, communities, institutions or society, or a combination of these levels? (Examination of logic model.)

Response:

5. Is it clear who the ultimate beneficiaries are, and what the intended impact will be? Cite the project final goal (or wherever the beneficiaries are specifically described).

Response:

6. Have assumptions (external factors critical to the achievement of outcomes but not within direct control of the project itself) been clearly identified? (Examination of logic model.) Response:

7. Is the project hypothesis (that outputs will lead to effects and desired impact) clearly stated and verified? Make reference to documentation.

Response:



Comment:

6. set a significant, yet achievable and measurable final goal. A project final goal must be achievable and measurable during the life of the project. This calls for project designers to clearly define what the project will be held accountable for achieving. It should be practical and do-able, yet be at the outcome level (intermediary impact or at least effect) rather than output level.

A project final goal must also be clearly and explicitly linked to, and significantly contribute to, higher level programme or strategic goals. Programme goals should address underlying causes of poverty and social injustice, but their impact equitable and durable improvements in human wellbeing and social justice should be ultimately manifest at the household or individual level.

1. What is the final goal of this project? (It should have been cited in the logframe, see Standard #5 above.) Is it significant, in that it addresses desired improvements in the lives of real people? Can it be said to address at least intermediary impact level? Response:

2. Is the final goal achievable and measurable during the life of the project? (This should be evident from the objectively verifiable indicators in the logframe and plans for measuring them, as in the M&E plan.)

Response:

3. What higher (program) impact goal does this project contribute to? Cite.

Response:



Comment:

7.be technically, environmentally, and socially appropriate. Interventions should be based upon current best practice and on an understanding of the social context and the needs, rights and responsibilities of the stakeholders.The project must be designed in a way that is likely to make a significant and positive difference, with minimal undesired social or environmental consequences. Interventions must make reference to technical or sectoral experience or standards, developed by CARE or others, to demonstrate the viability of their approach. Environmental analysis could include assessment of current status, analysis of potential impact, and regional environmental issues. These may require technical appraisal by those with expertise in the relevant professions.

1. Does the design of this project conform to current best practice guidelines for the relevant sector(s)? If so, what explicit reference has been made to such guidelines, or to lessons learned from evaluations of previous projects or research?

Response:

2. How well do you think this project meets sectoral technical current best practice guidelines?

Rating score for Standard #7-a: 1 2 3 4 - 53. Has there been an assessment of the potential impact of this project on the environment? Make reference to documentation.

Response:

4. How well do you think this project has considered environmental impact?

Rating score for Standard #7-b: 1 2 3 4 - 55. Did the diagnosis and design analyze and consider the social context, including the needs and rights of the intended beneficiaries and responsibilities of duty bearers? Cite reference.

Response:

6. How well do you think this project has considered human rights and other social impact?

Rating score for Standard #7-c: 1 2 3 4 - 57. Please add a comment on why this project may not currently meet this standard as well as it should, and/or what will be done to more adequately meet the standard in the future.

Comment:

8.indicate the appropriateness of project costs, in light of the selected project strategies and expected outputs and outcomes. Programme designers must be able to defend the budget of a project relative to its outputs, scale and anticipated impact. Also, the M&E plan should include methods for measuring cost effectiveness, i.e. to demonstrate that the costs of project interventions are reasonable and commensurate with the outputs and outcomes achieved.

8. Has any form of cost-effectiveness assessment been made, to estimate the relationship between the proposed budget and the intended outcomes (benefits)? (Include both the amount of planned change and the scale, i.e. how many people are likely to benefit, in what way?) If so, briefly describe or make reference to the results of such an assessment.

Response:

8. Has there been a comparison between the cost and planned outputs of this project to similar projects, or alternative interventions? Describe or cite reference in project documents.

Response:



Comment:

9.develop and implement a monitoring and evaluation plan and system based on the logical framework that ensures the collection of baseline, monitoring, and final evaluation data, and anticipates how the information will be used for decision making; with a budget that includes adequate amounts for implementing the monitoring and evaluation plan.M&E plans should provide sufficient detail to clearly identify evaluation design, sources of data, means of measurement, schedule for measurement, data processing and analysis, dissemination of information to and utilization by key stakeholders, and responsibilities for each of these processes. Sufficient budget should be allocated for designated tasks, and planning should ensure that CARE staff and partners have the capacity required for their implementation. Monitoring information should be useful and timely to promote reflective practice, for management decision-making, and for adapting project approaches and strategies. M&E plans should incorporate methods to measure risks and assumptions and to track unintended effects.

1. Is there a life-of-project M&E plan? How well does it include each of the components listed, i.e. evaluation design, sources of data, means of measurement, schedule for measurement, data processing and analysis, dissemination of information to and utilization by key stakeholders, and responsibilities for each of these processes?

Response:

2. How much is earmarked in the budget for M&E events and systems? Will this be sufficient to carry out what is called for in the M&E plan? Response:



Comment:

10.establish a baseline for measuring change in indicators of impact and effect, by conducting a study or survey prior to implementation of project activities. There needs to be a distinction between a diagnostic assessment and a baseline study. The former gathers a little information about many conditions and is used to inform project design. A baseline study, on the other hand, should focus on measuring indicators of effect and impact with a level of rigor required for a before-and-after comparison with evaluation. Baseline studies can use qualitative as well as quantitative data, as long as they describe the initial situation with sufficient precision to be able to clearly measure changes over the life of the project. 1. Is there a plan to measure impact and effect-level indicators at the beginning of the project, in such a way that the baseline data will be comparable with the final evaluation? Make reference to the plan for the baseline study.

Response:

2. (If the project is already underway:) Was the baseline study conducted on time (i.e. just before interventions were initiated)? Have the findings been used to refine the project plans and tighten up the project M&E system? Cite documentation.

Response:



Comment:

11.use indicators that are relevant, measurable, verifiable and reliable. Indicators should be capable of yielding data that can be disaggregated to the individual level according to criteria that reveal vulnerabilities, such as gender, age and social class. Both qualitative and quantitative measures are acceptable as long as they can illustrate discernible and significant change. For indicators to be reliable denotes that they are robust and will be useful and credible throughout the life of the project. CARE should draw upon the international development communitys great wealth of experience with indicators.

The following questions should be addressed after examining the logframe and/or M&E plan:1. Do the indicators in the logframe / M&E plan represent the appropriate level (e.g. impact indicators at final goal level)?

Response:

2. Are the indicators really objectively verifiable (i.e. unambiguous and precise in what needs to be measured or described)?

Response:

3. Can the indicators be disaggregated to distinguish changes among specific vulnerable sub-groups in the community (rather than simply the average of the whole population)?

Response:



Comment:

12.employ a balance of evaluation methodologies, assure an appropriate level of rigor, and adhere to recognized ethical standards.Evaluation should be incorporated as standard practice as a basis for accountability and for documented, institutionalized learning. Although various forms of evaluation should be planned, such as internal or external, formative (mid-term) or summative (final) or even ex post (to evaluate sustainability), the minimum is that there should be at least a final evaluation that summarizes the achievements and lessons learned by the project. Diagnostic assessments, baseline studies, monitoring, and evaluations should utilize a balance of methodological approaches to ensure triangulation, a richness of data, and mutual modifications. Evaluations should assure appropriate levels of rigor and precision in their designs and selection of methodologies. Informant confidentiality should be protected. Each evaluation event should draw upon previous ones and anticipate subsequent events. Evaluation processes must be documented and carefully archived, allowing subsequent project phases to replicate methods and draw upon comparative data.

The following questions should be answered after examination of the M&E plan:1. Does the M&E plan contain an evaluation plan that includes key questions to be answered, evaluation design (with or without baseline, with or without comparison group), and schedule for evaluation(s)? Cite.

Response:

2. Does the evaluation plan call for an appropriate mix of methodologies? What are they? Cite.

Response:

3. Is the evaluation plan based on a level of rigor appropriate to the required sophistication and precision required for measuring the achievements of this project? Is this even addressed in the plan? Response:



Comment:

13.be informed by and contribute to ongoing learning within and outside CARE.It is critical that relevant research and previous project evaluations inform the initial proposal preparation stage. More than that, learning should also apply throughout the life of a project and beyond. The lessons learned from a project should be adequately documented for utilization in the design of other projects. Project management should support the documentation of project processes, including re-designs. Reflective practice, such as the regular use of monitoring data, should be built into every project. Learning should be an organization-wide priority supported by frequent meta-evaluations.

1. Cite or make reference to evidence in project documents (design / proposal) that lessons learned from evaluations of previous similar projects and/or relevant research was incorporated in the plans for this project.

Response:

2. (If a new proposal:) Are there plans for documenting lessons learned during the life of the project? (If implementation is underway:) Have project processes, especially any changes in project design, been documented (e.g. included in annual reports or formative evaluations)?

Response:



Comment:

1 Original draft developed by CI Design, Monitoring & Evaluation Advisory Committee (DMEAC) in Atlanta 10/99; this revised version based on recommendations by IEI-II workshop participants in Wood Norton, UK, 31 August 2001 and follow-up communications. Final version approved by CI Programme Working Group (PWG) 5 April 2002 for recommendation to the CI Board in May.2 These standards refer specifically to CARE projects (whether implemented directly or through partners). However, where there are specific longer-term programme plans these standards should apply to them as well.3 This should not be taken literally to mean and be restricted to the classical 4X4 LogFrame matrix; rather more broadly interpreted to refer to logic model a summary graphical or matrix depiction of the logic of the project design.

care project standards measurement instrument

Documents