Value for Money

CARE UK DFID PPA Independent Progress Review

BackgroundCARE UK (CIUK) has secured £3.2 million per year through its 2011-2012 Program Partnership Arrangement (PPA) with DFID. These funds are focussed in the thematic areas of governance and social accountability, private sector engagement, climate change adaptation and peace building across selected CARE Country Offices (COs), with a primary objective of supporting the testing, refinement and promotion of innovative approaches to development programming, which is expected to improve development practice across CARE and the development sector more broadly.

Under the terms of the PPA agreement, CIUK commissioned a team of three independent consultants1 who conducted a mid-term evaluation (or “Independent Progress Review”) of the progress so far in the PPA. The Progress Review is also expected to assess the “value for money” (VfM) achieved through CIUK’s used of PPA funds as one of four primary objectives for the evaluation.

Defining “Value for Money”Value for money is defined in various ways as demonstrated by the range of definitions presented here:

1. “VfM is defined as the optimum combination of whole-of-life costs and quality (or fitness for purpose) of the good or service to meet the user’s requirement.” (HM Treasury2, 2006)

In this definition, the VfM services supported by the PPA would be assessed not solely on the basis of their cost, but also their “quality” relative to the needs of the “user”. Although the notion of quality is highly subjective, with this PPA it would involve the improvement of programming models that attracted wider use. Defining the users of PPA services is relative and requires clarification: CIUK’s PPA programme is not about direct implementation of activities but rather about promoting the innovation-testing-scaling process. Users of CIUK programme services are therefore defined as being primarily Country Offices and CARE International members, although the ultimate beneficiaries are obviously the impact groups which the development programming is expected to benefit and, as well, many of the tools and methodologies developed through the PPA would benefit the wider international development community. From the perspective of DFID’s theory of change, which sets the basic rationale for the PPA, the donor is also a user of CIUK services.

2. “...effectiveness and value for money are inextricably linked... effectiveness involves achieving a sustained impact for intended beneficiaries; and value for money is the best use of resources to deliver the desired impact.” (Independent Commission for Aid Impact3 ICAI, 2011)

1 The team is affiliated with Channel Research: Jups Kluyskens in the role of Team Leader, Judith Kunert as Project Manager and Steve Perry engaged as a consultant. Steve took the primary lead on this VfM analysis.2 The “Value for Money Assessment Guidance” released by HM Treasury in November 2006 was an early attempt at defining the VfM concept as it relates to UK government procurement processes, specifically under the Private Finance Initiative seeks to fund public infrastructure projects with private capital. While a useful starting point, this guidance is not strictly applicable to DFID’s programs.3 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. Its focus is on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers through independent reviews of aid programmes and of issues affecting the delivery of UK aid.

CIUK’s PPA design points towards the “intended beneficiaries” being, again, Country Offices who undertake programming that ultimately benefits impact groups and CARE International members. Evidence of “sustained impact” of the PPA for these beneficiaries would be observed when the programming models initially supported by the grant were subsequently built upon, expanded or leveraged outside the context of the PPA. Here the idea is that the targeted PPA investment in CIUK technical support services and the associated financing of what amounts to action research results in products that are useful in other contexts. Examining what the “best use of resources” is to achieve this result is intrinsic to the ICAI’s notion of VfM although this would be very contextual, related to how far along the development of programming models in CIUK’s thematic areas has moved, as well as the complexity of the Country Offices’ portfolios.

3. “A management approach to value for money: assessment of the extent to which key management processes and resource allocation decisions made at each stage of the implementation process results in the efficient delivery of higher value inputs, activities, outputs and ultimately outcomes and impacts.” (Coffey4, 2012)

This management approach is non-committal about the actual component pieces that make up the PPA logical framework, and is focused instead on how well DFID and CIUK (and in effect CARE more widely) are steering the program process “efficiently”. Efficiency is defined as the relationship between outputs – in terms of goods, services or other results – and the resources used to produce them. Efficiency in this sense considers budget magnitudes in relation to results achieved. This management approach then examines the decision-making frameworks around which these budget envelopes are determined, and their results assessed.

These three different notions of the VfM concept remain very relevant to this IPM exercise. However balancing the three definitions, the core idea of VfM is that resources are used optimally to reach the desired outcomes5. An approach to assess VfM is defined in more detail below.

The IP Review’s Approach to Assessing Value for MoneyThe core idea of VfM is that resources are used optimally to reach the desired outcomes. According to its logical framework, the CIUK PPA has two desired outcomes:

1. To enhance development impact in five sectors6 through promoting good practice and wider take-up of CARE innovative programming models within CI and amongst the development community; and

2. To enhance development impact by implementing innovative programming models in five specified sectors.

Innovation, promotion of good practice and wider take-up of successful programming models are intrinsic aspects of the CIUK PPA’s VfM proposition. Therefore it can be said that CIUK PPA funds expect to deliver value for money by:

Supporting processes that facilitate the packaging and promotion of innovative approaches/models for broader application and promotion. Without this additional investment, broader adoption of innovations is unlikely.

4 Coffey International Development are DFID’s consultants managing the overall PPA and GPAF evaluation processes considering the £480 million in funding to approximately 230 CSOs as a whole. 5 This is the definition of VfM as presented by the UK National Audit Office (NAO).6 Governance (including, Governance in the Maternal Health Context); Private Sector Engagement, Climate Change Adaptation and Peacebuilding/Conflict Sensitivity.

Reaching a much bigger audience than would otherwise be possible. This is achieved by investing relatively small amounts of resources to fine-tune innovative models. Successful models are then promoted for potential broader scale-up and adoption by CARE and other development actors, leveraging other funds and potentially multiplying the impact of DFID funds.

Improving the organisational effectiveness of CIUK, a key DFID partner, who administered total of £17.2m of other DFID funding in 2011-2012. Improvements in programming models supported by the PPA benefited this wider portfolio.

Adding learning to the development sector through “open source” materials created in collaboration with leading UK agencies.

PPA resources are actually spent by CIUK to achieve these outcomes in four broad categories of expenses:

Flexible funding provided by the PPA to directly support selected Country Offices in defined priorities related to the five sectors. These are the flexible annual grants.

Resources spent flexibly by the five thematic teams in initiatives that contribute to the development of programming models. These are the budgets managed by CIUK.

Compensating the time of CIUK technical specialists who support CO and UK-based activities. The costs of about twenty CIUK Advisors are substantially covered by the PPA.

Overall shared operational costs of CIUK, where these resources are managed. This is a fixed percentage shared by all grants covering overhead costs of CIUK’s operations.

The VfM equation therefore needs to assess the magnitude, efficiency and quality of these resources spent in relation to the achievement of the two desired outcomes. Therefore to test whether PPA funds are delivering value for money, CIUK needs to demonstrate7:

1. That innovative models/approaches/concepts tested by CARE are adopted on a broader scale – whether internally or externally8

2. That additional “flexible funding” is critical to this process. 3. That apart from the relatively unrestricted PPA mechanism, there are no other ways of

resourcing the development and packaging of innovative models for broader adoption9

4. That the resource levels invested in model development processes are “reasonable”5. That there are management mechanisms in place to steer the appropriateness and/or need

for the innovation investment10

The answers to these five questions are the core of the VfM assessment.

7 These five questions were proposed by CIUK.8 This is critical. Illustrative of this process would be where advocacy around child malnutrition by CARE Peru has resulted in a mainstreaming of recommended approaches by the Government into a national strategy – therefore reaching many more children than CARE could have done with its own resources. 9 If there are other methods to achieve the same result with other funding mechanisms the review will look at relative effectiveness10 This relates Coffey’s premise that management processes are important in the assessment of VfM Risk taking per se is not discouraged in the value for money discourse. When taking risks, some experiments will inevitably fail. However there should be management processes in place to mitigate or minimize risk in order for the mechanism to be considered good value for money

Methodology

In order to investigate VfM, the evaluators investigated CARE’s ability to innovate, pilot new models

and take these to scale at the CO level. The team looked at three Country Offices cohorts

differentiated on the basis of their access to flexible funding and technical support. In 2011 CARE

International supported seventy-two COs in developing world supported by twelve national

members, several of whom also manage operations in these COs11. The access COs have to flexible

funding and technical assistance varies quite widely depending on several factors including which

national member is providing the management services to the CO. The three CO cohorts or “pools”

were:

CIUK focus Country Offices The review team visited a sample of four COs12 and interviewed by telephone key informants from the other COs receiving support13. Based on the completeness of interviews and input received four CIUK focus countries were profiled: Peru, Bangladesh, Ethiopia and Sri Lanka. Each of these COs receives significant unrestricted resources from their management lead member. In addition, they receive flexible PPA funding and technical support through CIUK advisors.

COs which receive a significant annual allocation of flexible “unrestricted” funding from their lead member, although no PPA resources from CIUK The purpose of looking at this sample was to assess the particular value addition of CIUK involvement, and to assess the extent that innovation of new models and risk taking can occur where the pool of resources available to invest in this area is smaller or has different characteristics. Interviews took place with key CARE CO informants from Mozambique, Pakistan West Bank/Gaza and Vietnam14. These interviews focused particularly on the sources of technical assistance and knowledge management mechanisms in the five specific sectors (as these offices benefit from unrestricted resources, they have a different kind of access to the innovative models and lessons learned through the PPA’s innovation cycle).

COs that receive more limited “unrestricted” resources Key informants from Cuba, Zambia and Central America were interviewed.15 Similar to the second cohort, the reviewers were trying to determine how innovation took place in these CO contexts, and how this was financed.

With the COs that do not receive CIUK funding, no data was collected having to do with the question of how “reasonable” the resource levels were in the development of new innovative models. The annex summarizes the findings of this inquiry.

11 CARE USA, CARE Canada and CARE Australia are members who also lead management processes at the CO level. In addition, CARE France and CARE Germany also manage selected CO operations. As the “lead member” of a CO, these members are responsible to provide line management, administrative services such as international HR services, and are financially liable for obligations the CO incurs. The lead member also supports program development in the COs they manage, and this can include providing “unrestricted” funding to the COs for this purpose.12 Peru, Bangladesh, Rwanda and Somalia.13 DRC, Ethiopia, India, Ghana, Nepal, Bolivia, Sri Lanka and Uganda. Despite repeated attempts, the team could not make contact with key informants in Afghanistan or Yemen, possibly because of the timing of the exercise which fell during the month of Ramadan.14 Country Director or Assistant Country Director/Program Quality or Program Director15 CARE Zambia and CARE Cuba are lead managed by CARE Canada, and CARE Central America by CARE USA. The Central America mission has a unique structure where a single Director manages activities in three countries: Guatemala, Nicaragua and Honduras.

Value for money assessment This section answers the five CIUK questions or “tests” posed by CIUK to assess value for money, summarizing the interview findings with key informants. The section that follows considers the wider impact of PPA contributions to CARE.

1. Were innovative models/approaches/concepts tested by CARE adopted on a broader scale?16

The review has found numerous successful examples of the innovation-testing-scaling process taking place within CIUK partner COs which receive PPA 4 funding. However we note that other COs that do not receive PPA funding also consider innovation-testing-scaling as a modus operandi. Key informants in CARE Cuba and Vietnam specifically mentioned that sparking innovation and testing “new ideas” was now an expectation for INGOs by government counterparts. This process of scale-up is also an intrinsic aspect of the “programme approach” that CARE implements globally.17 The real “prize”, of course, is not simply securing a follow-up grant to expand successful activities in a particular CO - and this is where CIUK has a particular role to play - but the encouragement of broader national, regional and global dissemination of these innovations. Models/approaches/ concepts proven in one context need to be reworked for broader application. There is evidence that PPA-funded work has led to the development of consortium learning and capacity building programs (funded outside the PPA) such as the DFID conflict sensitivity consortium18 and the European Community DME project19, both of which have policy advocacy components which further extend their reach.

2. Is additional “flexible funding” critical to innovation-testing-scaling process?Flexible funding plays a critical role in this process. By its nature the innovation process is very difficult to plan for and budget given its unpredictability. Key informants report that flexible funding is critical in several areas: retaining key staff and their knowledge when their project funding ends, building relationships with partners and convening coalitions, undertaking wider contextual or cross-cutting analysis and strategy development, establishing the evidence base (especially ex post) of the innovation’s impact, and effectively communicating these results in order to establish CARE’s credibility in the new area/field.

CIUK’s sustained funding effort through successful PPAs has also been critical to this innovation-to-scale process, where a significant return on the PPA investment has only been realised after several rounds of funding. In Peru a detailed analysis of investments in the area of health was undertaken20 tracing the programme’s evolution from PPA 2 to date. Results and events that occurred along this timeline confirm the importance of using this semi-restricted resource in a strategic manner to build on the achievements of earlier investments, and, critically, to take advantage of opportunities that could not have been anticipated in the original design.16 Scaling-up is a critical aspect of the value for money proposition CIUK is making, however the process could be undertaken by CARE or other agencies. CARE Peru’s strategy looks to the Government to incorporate the approaches it advocates, therefore having a much greater impact than one organization working independently could ever achieve in the traditional project-based approach. 17 CARE USA, “Brief #1: What is a programme approach?” (2008).18 PPA support in the area of conflict prevention and peace building as far back as 2002 in PPA 1 led to the award of the “Conflict Sensitivity Consortium project” (2008-2012 £2.1m DFID funded) to promote understanding, capture lessons and build capacity in the area

of conflict sensitivity across 32 agencies and 4 countries. 19 The Peace building Design Monitoring and Evaluation project (2009-2011 €959,500 EC Instrument for Stability) is intended to critically review the “theories of change” being applied by peace builders in three countries and build capacity in their use in M&E. 20 Annex 2 is a visualisation of how the Health Rights Program in Peru evolved over successive PPAs into the Participatory Voices project.

Focusing the PPA funding on particular sectors and themes also provides good value for money. While the funding is appreciated as being highly flexible by COs, CIUK has been able to effectively complement the CO field work with its own global technical contributions over time, which has further promoted the scaling-up process. Examples of this are found within the development of governance models, conflict sensitivity tools and approaches, and the on-going work in the area of financial inclusion.

3. Are there are no other ways of resourcing this process besides using the relatively flexible PPA 4 resources?

PPA flexible grants to COs are not the only means of resourcing the process. Several kinds of unrestricted and flexible funding resources are available to all CARE COs to varying extents. COs managed by CARE USA (including most of the PPA focus countries21) all receive an annual unrestricted grant of between $200,000-1.2 million per year22. In all cases, some portion of this funding is used for programme innovation. Some CIUK COs benefit from flexible resources provided by other CARE members: Denmark, Australia and Canada each have programme funding similar to a PPA that is made available to selected COs. Corporate and foundation resources are also frequently used to support innovation and knowledge sharing. Excellent examples of this are the Barclay’s “Banking on Change” programme, or various initiatives supported by the Bill and Melinda Gates Foundation.

Where particular and unique value addition occurs through the PPA is through the role of CIUK, and its provision of technical backstopping and support. The PPA is distinct from other flexible funding sources in that it supports clearly distinguishable outputs that contribute in a strategic way to wider programmatic objectives. Key informants in COs indicated that this contribution greatly accelerates the process of innovation and scaling-up. PPA is different from the unrestricted grants provided by CARE USA because they are tied to particular programme themes, and this provides structure and discipline in their use. PPA funds are very flexible (COs can propose a very creative range of activities for support) but they cannot normally be used to cover operational budget gaps, which distinguishes the PPA from other types of flexible funds. CIUK focus countries that are part of the PPA have the expectation that the funding will continue; this appears to influence their decision-making in terms of how to spend the flexible funding. The reviewers see evidence that PPA resources are invested in “foundational” initiatives, such as programme analysis, strategic planning and the development and promotion of business plans.

Even beyond the PPA focus countries, key informants indicated that CIUK technical support was invaluable to CARE. CO Representatives in Pakistan, West Bank/Gaza, Mozambique, Central America, Cuba, Zambia and Vietnam all specifically recognized the usefulness of the tools, methodologies and documentation made available by CIUK technical teams throughout the CI network.

4. Are the resource levels invested in the process “reasonable”?The resource levels are reasonable. This review did not have the scope to investigate on a case-by-case basis, but the team did see sufficient evidence to assert that the PPA’s return on investment is

21 Peru and India were both CARE USA managed COs but both have begun the transition to independent CI membership. During this transition period, both offices continue to receive “core funding” that can be used to innovate and scale-up approaches and this is the dollar amount found under the unrestricted column for these COs in Annex 1.22 The CARE USA International Programme Operations department provided details of their unrestricted allocations over the FY12 and FY13 funding cycles to this evaluation. See Annex 1.

very significant in terms of the additional resources it leverages. CIUK programme £3.2 million per year through its PPA. £1,880,000 was budgeted for activities during the first year which was focused on four thematic areas (with direct expenditures on activities worth £660,000) across fourteen COs (who budgeted £900,000 for activities). The balance £1,350,000 was used to support the CIUK programme implementation costs, including the PPA’s share of Advisory personnel costs.

The strategic choices made by CIUK in terms of programme and geographic focus are reasonable, indicating that an even greater focus would develop the resource levels further. It is noted that CIUK work with climate change adaptation contributes to the mandate of the Poverty, Environment and Climate Change Network (PECCN), providing a contribution to its core costs as well as supporting specific thematic activities as well as initiatives at the CO level. CIUK acts as a resource for thematic technical expertise within CI in the areas of good governance, peace building/conflict sensitivity mitigation, private sector engagement and climate change adaptation. The cost of providing this technical expertise (£1.35 million per year) is extremely modest considering the breadth of CARE International’s global portfolio. By CIUK’s best estimate, 80 percent of its technical teams' costs across the four thematic focus areas are met by the PPA.

5. Are management mechanisms in place to steer the process and mitigate risk?It is the opinion of the evaluation team, based on field visits and inquiries at the level of the CO in four countries, that these two management levels adequately mitigates the risk of inappropriately or inefficiently investing resources. Risk taking per se is not discouraged in the value for money discourse; when taking risks, some experiments will inevitably fail. However, for there to be value for money, there should be management processes in place to mitigate or minimize this risk23. The review team observes risk management at two levels; field and CIUK headquarters.

At the CO level, a number of management approaches help to mitigate risk:

CO programme (within which PPA activities are implemented) are designed within a strategic framework overseen and approved by regional management.

All CARE COs have standardized financial and administrative compliance processes in place to ensure that resources are allocated efficiently and effectively, for example in terms of procurement or contract management.

Particular attention is paid to compliance issues where they relate to partnership and utilizing PPA resources. COs follow standardized guidelines to assess the capacity of organizations and provide sub-grants on the basis of written agreements with partners.

Checks and balances exist in the project design process, often involving line management working in conjunction with an assistant director for programmes, who often works in close consultation with CIUK.

Many COs have experienced monitoring and evaluation teams which provide insight into field implementation processes, including learning and cross-fertilization of experiences.

Second, risks are also managed by the CIUK technical teams in the four thematic areas. In the first year of PPA COs developed their planning proposals iteratively with their Advisor counterparts in CIUK, and these were presented by the thematic Team Leaders to the PPA Committee. Although CIUK reported that this system worked well, for second year proposals the PPA Committee is no

23 This test is aligned to the Coffey “management approach to value for money” approach.

longer directly in the decision loop but is able to intervene and steer proposals where the committee perceives the need for additional refinement.

Impact and value for money of PPA fundingPPA-funded results have produced significant value for money for CARE International and COs working on behalf of their impact groups. The impact of innovations and their relationship to CARE’s programme approach is a key aspect of the VfM proposition.

Summarizing the impact the programme innovations have had is difficult due to the highly pixelated nature of the PPA. The PPA is characterized by numerous small, targeted expenditures. Attempts have been made to formally estimate the return on investment of particular innovations, such as the community development forums in Sri Lanka, which concluded that this activity had a very high “social return on investment”24. Given the wide diversity of the initiatives themselves, as well as the contexts where they work, an impact summary for PPA 4 is impossible to be reached conclusively, especially at this early stage of the funding cycle. However, in this review we have considered the PPA’s impact over time as well as its impact as a wider learning programme and we have concluded that it has had a significant impact on both CARE and the COs.

The thematic areas which CIUK has invested in are also critical in CAREs advance towards a programme approach. In 2008, around the beginning of PPA 3, CARE International began a process aimed at achieving more sustainable impacts in terms of the underlying causes of poverty and social injustice. The timing of this was fortuitous as it created a significant space for CIUK to provide value addition in a highly meaningful way to the global development agenda of CARE International. At this point through its own strategic planning process CIUK identified its focus areas through its identification of specific thematic areas where it would focus and provide technical backstopping. PPA resources allowed CIUK to invest in these thematic areas by investing in the innovation-testing-scaling process around these themes, while at the same time working at the global level in the development of tools such as the Governance Programming Framework. CIUK became known for the high quality of its technical assistance. This has continued through PPA 4 as this “programme shift” continues and the CARE International business model continues to evolve.

Several field-based key informants made similar observations concerning various types of PPA technical support they benefit from:

The flexibility around the annual planning process involving the CIUK Advisor and the CO is appreciated. It allows for a meaningful dialogue around which programme priorities can take place. This iterative process also provided useful encouragement to align field activities with CIUK’s theory of change around developing innovative models for broader application and promotion. For example such a “push” in FY12 has moved CARE India away from directly implementing its micro-finance work towards business planning analysis; this is more useful to partners in the eventual scale-up of this innovation.

CARE can reach a much bigger audience than would otherwise be possible. COs report that they use their small, targeted PPA investments to amplify the impact of a particular tool or finding. CARE 24 The project where the community development forum modality was studied in Sri Lanka generated a social return on investment of 8.52%, with returns accruing to the State and private sector as well as the communities involved. “Desk review to conduct assessment of ‘value for money’ provided through CARE International’s programmes to vulnerable and marginalised populations in Asia”, NEF (new economics foundation), December 2011.

Ethiopia’s experience with governance and accountability tools in PPA 3, where it broadened the use of community score cards, influenced its overall approach to local government and in both the area of health and food security, as well as populace livelihood. They hope investments in the area of climate change adaptation tools in PPA 4 will bring about a similar result. At the CO level resources are used to leverage change across the entire portfolio, rather than improving isolated “project islands”.

Communities of Practice (CoPs) offer field staff a ‘window’ into what other COs are doing. Several of these CoPs are coordinated by CIUK Advisors. The conflict CoP has connected field staff from various COs and has promoted experience sharing. According to one ex-CoP coordinator, the level of effort required to facilitate this process requires dedicated staff time, without which this investment in the CoP platforms would stagnate.

Materials and tools developed by CIUK are widely shared, and CIUK Advisors are frequently engaged in strategic planning processes that benefit CARE International more broadly. These contributions outside the parameters of its particular geographic focus are a significant addition in terms of monetary value from CIUK’s membership role within the confederation. More than one key informant in a non-CIUK CO recognized that Advisors are invariably responsive to their emailed queries on technical matters even though they have no specific “stake” in their portfolio.

At a broader level, technical contributions, tools and policy advocacy undertaken by the CIUK-based thematic teams are appreciated by the broader development community. Evidence of this was the inclusion of a CARE policy document as a reference in a UK government funding guideline. CIUK’s conflict team (largely funded by the PPA) developed influential tools in the conflict and peace building theme. The Foreign and Commonwealth Office referred applicants to the conflict pool for South Asia to a CARE guidance note.25

Attributable impacts of PPA funding on results, relevance, effectiveness and efficiencyGiven the programme’s design, it is not realistic to establish attribution between PPA funding and impact. The innovation-testing-scaling process that the PPA supports includes contributions from many directions. It is carried out efficiently using a blend of human resources (both internal to CIUK, external experts and CO staff) through activities undertaken at the field level by COs and partner organisations. PPA contributions are generally added to programmes already in place either at the CIUK or CO levels, which are often funded by multiple donors and consist of multiple project activities. In the field context, PPA resources are considered extremely strategic by CO beneficiaries as they allow experimentation and risk taking to take place that would otherwise not happen within the constraints of a traditional project approach with defined deliverables and restricted funding. PPA resources are able to add value to these existing grants by developing innovations that improve their impact, or by strengthening the impact evidence base that allows CARE to promote or advocate for models that are particularly effective.

A particularly good example of the trickiness of attribution is found with the CARE Peru “Participatory Voices” activity, which has received PPA funding through several phases. At the

25 CARE International UK (2012): Peace building with Impact: Defining Theories of Change. This specific document was not PPA funded. It was developed using funds from an adjacent Peace building Design M&E Project supported by the EU, but the study drew on field work that was supported by the PPA.

impact level, the PPA’s log frame includes improvements in the rates of maternal mortality26. Our review concludes that it is reasonable to claim that Participatory Voices has made a significant contribution to global reductions in maternal fatalities in Peru, particularly in the regions where CARE is active. But obviously many other actors – including the Government’s health care system itself – also contribute to improvements in this area. Moreover, as Participatory Voices has received PPA funding since 2005 across three different programme phases, it is very difficult to attribute these impacts to a particular PPA investment during this period, as the programme has evolved and strengthened over this time, so that the results being measured today are a result of investments made earlier.

Value for money assessment of PPA fundingThis review has determined that CIUK deliver VfM in four ways: (1) facilitating the packaging and promotion of innovative approaches/models for broader application and promotion within and outside supported COs; (2) making small investments that yield big returns in terms of leveraged funding or other partners taking up the approaches that CARE advocates; (3) improving the quality of CO interventions in the focus theme areas which has a knock-on effect to the wider portfolio of activities; and (4) providing intellectual and technical leadership in focus theme areas which allows COs and CARE more widely to engage in areas such as climate change adaptation or engagement with the private sector where their experience is more limited. By using a control sample of COs to analyze how the innovation-testing-scaling process occurs in contexts that do not benefit from the PPA, it can be concluded that CIUK delivers substantial value for money.

The flexibility of this type of funding allows CIUK and COs to quickly innovate and exploit lessons learned and new opportunities. While innovations leading to this kind of impact can and do also occur in activities funded with restricted resources, the key informants this review interviewed, who have experienced working within and outside of the PPA report that the process is much slower when uses only project funding. Moreover, other key informants report that major barriers exist for CARE with regards to technical areas and themes where the CO does not already have credibility. With PPA funding it is possible for COs to invest in new areas, deepen experience and build credibility that can subsequently be leveraged to win restricted funding. Finally flexible funding allows CARE to expedite the development of its expertise base; this has broad positive impacts across CARE International.

CIUK could be more demanding of COs to even further increase its value for money. For example, the reviewers appreciate the management approach taken with the current PPA 4, but we understand from interviews in CIUK that in earlier PPA phases the programmatic steering was less rigorous. The review team sees some isolated cases where PPA resources continue to support the implementation of activities that are not clearly “new innovations”, but already tested approaches. The reviewers are not saying that any resources are being inappropriately used; rather we are challenging CIUK and COs to be more innovative and entrepreneurial in their approach, and more alert to the importance of appropriately harnessing the learning that is occurring. The reviewers see innovation, entrepreneurial risk and learning as central to the VfM proposition.

26 This log frame impact indicator is “Number of maternal fatalities in Peru at national level and in 3 priority regions (Ayacucho, Huancavelica, and Puno)”.

It has also been observed that a key distinguishing characteristic of the PPA is the advisory role of CIUK. To a great extent, it is this knowledge management function that sets the PPA’s flexible funding apart from other similar resources available through other CARE members. Amongst other duties Advisors organize learning events, manage methodological development and support innovation processes occurring in the field. Set in the position of “network hubs” for the different theme areas, it is incumbent on CIUK to maximize this learning that is shared to the extent possible within the CARE environment and the broader development community.

Overall impact and value for money of PPA funded activitiesCOs which received PPA funding all made similar observations concerning the effectiveness of this funding modality in terms of its improvement of their programme’s impact. The PPA allows these offices to more fully develop promising approaches and ready these for broader application and sharing. PPA resources are extremely strategic for these COs as they allow experimentation and risk taking to advance that which would otherwise not happen within the constraints of a traditional project approach with defined deliverables. PPA resources are able to add value to these existing grants by developing innovations that improve their impact, or by strengthening the impact evidence base that allows CARE to promote or advocate models that are particularly effective.

The review team spoke with a sample of COs who do not receive PPA funds report and finds that they universally wish they had access to them, as the global reputation of PPA within the organization is so positive. These key informants emphasise that the benefits of the programme is not just the money (which all offices could use more of) but the technical assistance and global relationships that CIUK facilitates and brokers. COs that currently are not part of the programme report that the tools and approaches developed and promoted using PPA resources (in the areas of governance, conflict sensitivity or private sector engagement) are widely used nevertheless, as CIUK makes these available globally within the CARE International confederation.

Annex 1: Value for Money dataPPA RecipientsKey Informants

UNR received 1. Innovations and scaling

2. Role of flexible funding

3. Alternatives to PPA funding and support

4. Scale of resources “reasonable”

5. Management processes in place

PeruMilo Stanojevich, CARE Peru Director NacionalAriel Frisancho, CARE Peru Coordinador del programas de SaludGaia Gozzo, CIUK Senior Governance Advisor

PPA (£125K), CUSA27 ($1.2 million)

PPA “LAC Annex” was much larger scale28. Work continues in area of maternal health/governance (with impressive results). Expanding into several PSE areas, including with extractive industries. Work in area of climate change on a wide and strategic level with World Bank and Global Environment Fund.

Critical to support the incremental and organic process where flexibility allows capacity to engage in opportunities as they arise. At the outset of an activity it is not possible to predict all the required results (e.g. ex post evaluation), and budget them into a program. Flexible funding allows team to advance one step at a time.

Sale of Financiera Edyficar S.A., a financial entity created by CARE that provided financial services to low-income sectors, has generated net assets that are now used in other areas.Success in particular technical areas has positioned CARE well amongst other organizations that have considerable technical capacity.

In the Health Rights Program era PPA resources were provided on a much larger scale.Targeted and strategic use of PPA resources now is extremely impressive.

Affiliate CI member. Continued relationship with LAC Regional Management Unit (or RMU, a CARE USA management structure). Independent Board.

Frequent interaction with CIUK LAC PSE Advisor based in Brazil

PECCN LAC Adaptation advisor regionally posted.

BangladeshAsif Ahmed, PSE DirectorSelim Reza Hasan, Director Climate ChangeAnowarul Haq, Impact Director

PPA (£150K), CUSA ($250K)

Long relationship with PPA has resulted in several initiatives developing in innovative and wider scale particularly in the area of PSE (rural sales program), and governance (Botlagari model). No specific innovations named in the area of climate change.

Supports small scale and strategic aspects of existing models (e.g. developing business plans and management consulting services to emerging enterprises). Additional analysis to generate learning and evidence, or consider the applicability of the model in other contexts.

With Rural Sales Program the need to invest in a social enterprise was not pre-planned and therefore not associated with a budget allocation. CARE Bangladesh works on a very large scale, and its size generates opportunities as projects can cross-fertilize lessons and approaches in a true program approach. There are not many other funding sources that provide

Given the variety and scale of initiatives the PPA is involved in, resource levels are reasonable. They represent a very small proportion of the overall budgets involved. Despite the size of the grant the number of specific initiatives is considerable.

Line managed by CUSA Asia RMU, receives support the Asia PSE Advisor (which is CIUK funded, but reports to the RMU)Internally “Resource Group” generates dialogue, debate and discussions on learning within the organization. Linked to CIUK Advisor. Impact Director and ACD Programme ensure that learning is shared and included in new programmes. Programme

27 CUSA-CARE USA. Data for CARE USA unrestricted allocations provided by the International Program Operations department.28 When DFID chose to phase-out its funding to Latin America and the Caribbean (LAC) it used the PPA as a means to wind-down activities in a more managed manner. As CARE had just won a large DFID funded Health Rights Program, it received additional support through the LAC annex to continue this work. The Health Rights Program later evolved into the current, PPA funded, Participatory Voices project.

PPA RecipientsKey Informants

UNR received 1. Innovations and scaling

2. Role of flexible funding

3. Alternatives to PPA funding and support

4. Scale of resources “reasonable”

5. Management processes in place

flexible funding to help integrate projects.

Quality Unit facilitates reflective learning processes.

EthiopiaGobena Seboka, Food Security Program Coordinator

PPA (£40K), CUSA ($313K)

Considerable success and scale-up of PPA3 governance tools (e.g. community scorecards). Intend to use climate change tools (e.g. CVCA methods29)

Training to extend and mainstream promising approaches and models. Bridge funding of key staff to retain knowledge when their funding stops.

Considerable regional resources exist in the area of climate change adaptation and food security, and CARE has considerable depth in the latter area.

Very strategic and targeted. CO will move forward in three gender-related areas related to climate change adaptation and food security this FY.

Line managed by CUSA East and Central Africa RMU.

PECCN regional resources available in climate change adaptation area

IndiaAndrea Rodericks, Program DirectorR. Devaprakash, Team Leader Livelihoods

PPA (£65K), CUSA ($620K)

Work following up on Tsunami response along eastern coast. Micro-insurance innovation to build “literacy” among the poor. Resiliency building strategy with wide relevance outside initial working areas in other programs.

Development of micro-insurance model. This has required considerable CARE intervention. With redesign of approach, CARE now focusing on learning and future, such as investing in a Business Plan that elaborates the framework and allows other partners to engage.

Considerable engagement with private sector insurance companies (Bharti AXA Life and Bajaj Allianz) but these current relationships have been built on the PPA’s foundations.

CO program strategy work has identified the question of resilience building as critical in the area of disaster risk reduction. In this context, continued investments in the micro-insurance areas will serve to maintain CARE’s profile in this area.

Affiliate CI member. Continued relationship with LAC RMU. Independent Board.

CIUK regional PSE Advisor based in region.

Sri LankaFaisal Cader, Program Director, Plantation Program

PPA (£30K), CUSA ($300K)

Community Development Forums now a central feature of a new EC grant. Model seen as relevant to plantation managers, who are supportive with their own resources.

Replication of models into adjacent areas; documentation of impact; innovative “social return on investment” study

The number of traditional institutional donors are diminishing, with some large donors phasing out. CIUK support is important, especially at the present stage of organizational evolution within CARE.

This modest investment has led to a significant reputation boost for CARE related to CDFs.

Line managed by CUSA Asia RMU

Received support from CIUK advisory resources, including M&E

Unrestricted Resources provided through other means than PPA VietnamPeter Newsum, Country Director

CARE Australia (A$200K), CARE Danmark30,

Replication of successful models is important but not done well. Need to connect better with government, a critical player in the

Project development (resource mobilization) and the communication and development of project models. ANCA grant used to

Aggressive in building learning and analysis into traditional project proposals. Advocacy with traditional donors to adapt

NA Line managed by CAust (Canberra). PECCN and CIUK technical staff based in Vietnam.

29 Climate Vulnerability and Capacity Analysis

PPA RecipientsKey Informants

UNR received 1. Innovations and scaling

2. Role of flexible funding

3. Alternatives to PPA funding and support

4. Scale of resources “reasonable”

5. Management processes in place

ANCA grant31 Communist system. With projects linked to government, CARE continues to provide TA to ensure quality.

support cross-cutting activities that benefit multiple projects.

their thinking to CARE’s learning. Successfully leveraged PECCN and PSE presence in Vietnam. Use of interns and volunteers.

PakistanWaleed Rauf, Country Director

CUSA ($350K)32 New office that has been beset by humanitarian disasters (earthquakes and floods) since it was opened. Scaling up of models from other countries (e.g. the Rural Maintenance Program from Bangladesh).

Project development and quality, but also match funding and general “shared costs” that are unsupported by restricted grants. Allows CO to move into areas where it does not currently have a reputation.

Relationships building with the private sector and discovering they innovate and move much faster than NGOs do!

NA Line managed by CUSA Asia RMU

West Bank GazaSarah Ralston, Assistant Country Director

CUSA ($200K) Biggest success maybe the “Seed bank” model which has spread widely beyond original investment area to include many additional cooperatives.

Historically used to cover “gaps” but a recent organizational redesign has freed resources allowing a focus on situational analysis and the “program shift”. Small unrestricted amounts used to leverage major changes and innovations (e.g. Seed banks)

TA from CUSA. NA Line managed by CUSA Middle East and Europe RMU

Central AmericaStephen Grun, Program Director

CUSA ($960K) Project model with parameters defined by institutional donors (e.g. USAID RFPs and RFAs) do not allow much room for breakthrough ideas. Changing role from implementer to facilitator that will promote

Not much flexible funding is available, and it is used to balance the books of four offices (Honduras, Guatemala, El Salvador and Nicaragua) which each have different funding contexts and cost structures.

TA from CUSA. NA Line managed by CUSA LAC RMU

30 DANIDA framework agreement supports work in eight focus countries (Niger, Ghana, Uganda, Kenya, Tanzania, Mozambique, Vietnam and Nepal). CO programs have defined results, but allow for – and encourage – considerable innovation.31 ANCA (Australian NGO Cooperative Agreement) are resources available to Australian-based NGOs which CARE Vietnam qualifies for as its management member is CARE Australia. This additional resource is used for analysis underpinning the various programs, and is used to fund cross-cutting positions in the area of gender, monitoring and evaluation and program quality.32 CARE Pakistan also has unspent emergency appeal money to support program quality initiatives related to emergency.

PPA RecipientsKey Informants

UNR received 1. Innovations and scaling

2. Role of flexible funding

3. Alternatives to PPA funding and support

4. Scale of resources “reasonable”

5. Management processes in place

scalability.MozambiqueRené Celeya, Country Director

CUSA ($450), CARE Denmark, Fall Family Foundation33

Core program area was water and sanitation. Restricted funding now channelled into local government implementation of approaches promoted by CARE.

Five fold: (1) promotes strategic directions and innovative risk taking; (2) organizational priorities (such as “talent management”; (3) shared costs; (4) proposal development; (5) CSO strengthening and relationship development

TA from CUSA (e.g. Economic Development Unit, food security team), PECCN (Adaptation and Learning Program), CARE Denmark. Support from private foundations interested in supporting “the right thing” in terms of innovation and learning.

NA Line managed by CUSA Southern Africa RMU

Effectively no unrestricted resources availableCubaChristina Polzot, Country Director

CARE Canada (C$25K)

All innovative ideas come from CARE Cuba’s CSO partners. Cuba is unusual in this way as CARE does no implementation or testing: very small office. CARE’s role to document and communicate these to attract investment money. Government is the only channel to take models to scale, and so has a critical role.

Unrestricted grant to support operational budget shortfalls only.

CO in a very isolated position. Cuban system is unique. Many program areas where CARE has comparative advantages (e.g. value chains and markets, or governance and accountability) are “off limits” because of the system. Visa and other restrictions make staff mobility for staff and partners difficult.

NA Line managed by CCan (Ottawa)

ZambiaJay Goulding, Assistant Country Director

CARE Canada (C$100K)34

Success in particular areas (e.g. urban water and sanitation) with models have led to larger scale replication.

Used to generate new proposals for restricted funding.

Funding from USAID comes with CUSA relationship. Other members also support proposal development in proposals they have a stake in. Innovation and learning built into restricted grants. Case-by-case support from CCan (e.g. gender equality

NA Line managed by CCan (Ottawa)

33 Provides $1-2 million per year to support an innovative collaboration between the World Wildlife Fund and CARE.34 Negotiated use of miscellaneous income (MIME) generated by CO that normally returns to CARE Canada Ottawa. Typically about C$100K is available per year.

PPA RecipientsKey Informants

UNR received 1. Innovations and scaling

2. Role of flexible funding

3. Alternatives to PPA funding and support

4. Scale of resources “reasonable”

5. Management processes in place

strategy). Use the CUSA “PAL” system (that directs small donors to specific projects) to support innovative ideas.

Annex 2: The Evolution of “Participatory Voices” in CARE Perú, 2003-12