people's republic of china rural finance sector programme
TRANSCRIPT
P R O J E C T P E R F O R M A N C E A S S E S S M E N T
December 2013
IFAD Internal Printing Services
People's Republic of China
Rural Finance Sector Programme
International Fund for Agricultural Development
Via Paolo di Dono, 44 - 00142 Rome, ItalyTel: +39 06 54591 - Fax: +39 06 5043463E-mail: [email protected]/evaluationwww.ruralpovertyportal.org
ifad-un.blogspot.com
www.facebook.com/ifad
www.twitter.com/ifadnews
www.youtube.com/user/ifadTV
Independent Offi ce of Evaluation
December 2013 Report No. 3041-CN Document of the International Fund for Agricultural Development
Independent Office of Evaluation
People’s Republic of China
Rural Finance Sector Programme
Project Performance Assessment
This report is a product of staff of the Independent Office of Evaluation of IFAD and the findings and conclu-sions expressed herein do not necessarily reflect the views of its Member States or their representatives to its Executive Board. The designations employed and the presentation of material in this publication do not imply the expression of any opinion whatsoever on the part of IFAD concerning the legal status of any country, terri-tory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. The designa-tions “developed” and “developing” countries are intended for statistical convenience and do not necessarily express a judgement about the stage reached by a particular country or area in the development process.
All rights reserved. ©2013 by the International Fund for Agricultural Development (IFAD)
Preface
China is the fourth largest country in the world and home to more than 1.3 billion
people. Since the start of far-reaching economic reforms in the late 1970s, China has
witnessed unparalleled economic growth that has fuelled a remarkable increase in per
capita income and a decline in poverty. The economy has performed well in recent years,
even against the backdrop of the worldwide financial crisis and soaring food prices.
Despite its remarkable progress in economic and social development and poverty
reduction, China still faces many challenges to reduce residual poverty. Disparities in
income among provinces and between urban and rural areas have been widening and
poverty remains primarily a rural phenomenon.
The purpose of this project performance assessment is to assess the results and
impact of the Rural Finance Sector Programme and gather lessons that would improve
the design, implementation and impact of future IFAD operations.
The project contributed to increased access to finance by the poor, women and
small producers. It also contributed to enhancing the livelihoods of rural households in
the project counties by an increase in household assets and production, improvement in
food security, and in the strengthening of human and social capital and socio-economic
status of women. However, the project originally designed as a sector project to
participate in systemic policy reforms could not do so due to a lack of proper institutional
arrangements, rapid changing nature of the ongoing reform process and inadequate
grasp of the reform issues and institutional complexity of the rural finance sector in
China.
Special thanks are due to the project management offices of Chongqing Rural
Commercial Bank and Shaanxi Provincial Rural Credit Cooperative Union, and senior
management and field officials of both organizations for their excellent cooperation, for
providing data and assistance without which it would have been impossible to complete
this assessment. The beneficiaries deserve special mention for sharing their views and
experience with the evaluation mission.
This project performance assessment was prepared by Mark Keating, Evaluation
Officer, with inputs from Dewan A. H. Alamgir, rural finance specialist consultant and
Linda Danielsson, Evaluation Assistant. Konstantin Atanesyan, former Senior Evaluation
Officer, and Anne-Marie Lambert, Senior Evaluation Officer provided comments on the
draft report. This Office is grateful to the Asia and the Pacific Division of IFAD for the
input and support provided throughout the evaluation process, as well as to the
Government of China for its constructive collaboration. The evaluation would like to
thank Sun Yinhong and Weijing Wang of the IFAD China Office for their excellent support
during the field visits.
Contents
Currency equivalent, weights and measures ii
Abbreviations and acronyms ii
Map of the project area iii
Executive summary iv
I. Background and methodology 1
II. The project 2
A. The project context 2 B. Project implementation 4
III. Review of findings 11
A. Project performance 11 B. Rural poverty impact 16 C. Other performance criteria 19 D. Performance of partners 22 E. Overall project achievement 23
IV. Conclusions and recommendations 23
Annexes
I. Rating comparison 26
II. Basic project data 27
III. Terms of reference 28
IV. Methodological note on project performance assessments 33
V. Definition of the evaluation criteria used by IOE 37
VI. List of key persons met 38
VII. Bibliography 39
ii
Currency equivalent, weights and measures
Currency equivalent
Currency unit = Chinese CNY
US$ 1.00 = CNY 6.3 (as of June 2012)
Weights and measures
1 kilometre (km) = 0.62 miles
1 metre (m) = 1.09 yards 1 hectare (Ha) = 10.000 m2 (0.01km2) 1 hectare (Ha) = 2.47 acres 1 acre (ac) = 0.405 hectares (ha) 1 kilogram (kg) = 2.204 pounds 1 pound (lb) = 450 grams (gr)
1 ton = 1000 kg
1 gallon (gl) = 3.785 litres (lt) 1 m3 = 35.3146 ft³ 1 m3 1000 litres (lt)
Abbreviations and acronyms
CBRC China Banking Regulatory Commission
CRC Bank Chongqing Rural Commercial Bank
GDP Gross Domestic Product
IFAD International Fund for Agricultural Development
MTR mid-term review
NPL non-performing loan
PCR project completion report
PCRV project completion report validation
PMO Project Management Office
PRCCU Provincial Rural Credit Cooperative Union
RCB Rural Commercial Bank
RCopB Rural Cooperative Bank
RCC rural credit cooperative
RCCU rural credit cooperative union
RFSP Rural Finance Sector Programme
RIMS Results and Impact Management System
UNOPS United Nations Office for Project Services
iii
Map of the project area
iv
Executive summary
1. The Independent Office of Evaluation of IFAD (IOE) conducted a project
performance assessment (PPA) of the IFAD-funded Rural Finance Sector
Programme (RFSP) in China in June-July 2012. The assessment comprised the
review of key project and other official IFAD documents, as well as other relevant
literature, and a short mission to China where the evaluation mission met with
project partners, relevant stakeholders and beneficiaries, and observed ongoing
activities following project closing. The RFSP was designed in 2002/03, approved
by the IFAD Executive Board in April 2004 and became effective in September
2005. The project completed all of its activities in March 2010, six months after the
original completion date, and submitted a project completion report (PCR) prepared
by the Ministry of Finance in 2010.
2. The project was designed at a time when the Government of China had been
carrying out a major reform of its rural financial sector. The RFSP concept
envisaged participation in the reform process by assisting the government to
introduce systemic changes in the rural finance sector. At the time of project
design, the rural finance sector in China was managed by government-owned,
underperforming networks of rural credit cooperatives (RCCs). The China Banking
Regulatory Commission (CBRC) spearheaded reforms that focused on
a) developing clear corporate structures for the RCCs at provincial and county
levels; b) developing mixed ownership (government and private); c) improving
profitability by enhancing operating efficiency and reducing non-performing assets;
and d) increasing access to credit. These objectives have largely been achieved by
converting many RCCs into rural commercial banks and cooperative banks.
3. In this context, the RFSP was designed to partner with CBRC to contribute to
reforming the rural financial sector. This was to be achieved by converting the
network of rural credit cooperatives into sustainable microfinance institutions that
are accessible to poor rural people, women in particular. The programme
comprised five components: (a) policy development; (b) institutional development;
(c) operational development; (d) financing; and (e) programme management.
4. Overall, the evaluation found that the project contributed to strengthening
institutional, financial, and operational capacity of the two implementing financial
institutions – Chongqing Rural Commercial Bank and Shaanxi Rural Credit
Cooperative Union network – that led to increased access to finance by the poor,
women and small producers. It also contributed to enhancing the livelihoods of
rural households in the counties covered by the project. Impact studies showed
increase in household assets and production, improvement in food security, and
strengthening of human and social capital and socio-economic status of women. At
the same time, the positive impacts on households cannot be fully attributed to the
project alone. The two financial institutions are institutionally and financially viable
to continue offering financial services to poor smallholders. However, the
participation of the project in the systemic policy reforms led by CBRC did not fully
materialize due to a lack of proper institutional arrangements, rapidly changing
nature of the on-going reform process and inadequate grasp of the reform issues,
and institutional complexity of the rural finance sector in China. The project
concept was relevant but design was ambitious and weak in terms of institutional
arrangements.
5. The evaluation noted that there is a need and scope for future reform in the rural
finance sector in China. A number of issues, such as: (i) lack of strong competition
in the rural areas; (ii) potential effectiveness of village and township banks as they
are initiated by RCCs or RCBs; (iii) ownership structure of various institutions and
prudential regulatory issues; and (iv) strengthening of governance are all areas
where further research and reforms are needed. The evaluation recommends for
v
IFAD to maintain regular interactions with CBRC and the Ministry of Finance to
explore possibility of participating in future rural finance sector reforms.
1
People’s Republic of China Rural Finance Sector Programme Project Performance Assessment
I. Background and methodology 1. This report presents the findings and recommendations of the project performance
assessment (PPA) of the Rural Finance Sector Programme (RFSP) funded by IFAD
in the People’s Republic of China. IFAD’s Independent Office of Evaluation (IOE)
conducted the PPA during June-July 2012. The RFSP was designed in 2002/03, it
was approved by IFAD’s Executive Board in 2004 and became effective in 2005.
The project completed its activities in 2009/10 and the Government prepared a
project completion report (PCR) in 2010 through the implementing agency, the
Ministry of Finance. IOE conducted a PCR validation (PCRV) of the project and
selected it as a sample project for conducting a full PPA.
2. The PPA is carried out to: a) provide an independent assessment of the overall
results and impact of the project for accountability and management purposes;
b) distil lessons learned identifying key explanatory factors of project performance
and poverty reduction results for learning purposes; and c) gather additional
evidence on the major information gaps, inconsistencies or analytical weaknesses
of the PCRV. Under the current IFAD Evaluation Policy of 2011,1 IOE undertakes
PPAs for selected projects instead of project evaluations as was done in the past.
The findings and recommendations also benefit the design and implementation of
ongoing and future operations within the country.
3. IOE conducts a PPA as a follow-on step after producing a project completion report
validation (PCRV) report through desk review of various documents, including the
PCR, official project documentation, impact studies and other informative literature.
The PCRV serves the following purposes: (i) independent verification of the
analytical quality of the PCR; (ii) independent review of project performance and
results through desk review; and (iii) extrapolation of key substantive findings and
lessons learnt for further synthesis and systematisation. A PPA includes country
visit in order to complement the PCRV findings and fill in information gaps
identified by the PCRV.
4. The PPA applies the evaluation criteria outlined in the IFAD Evaluation Manual.2 In
view of the time and resources available, a PPA is generally not expected to
undertake quantitative surveys. Based on the initial findings of PCRV process, the
PPA rather adds analysis based on interviews at IFAD headquarters, individual
interviews and discussions with stakeholders in the country including project
beneficiaries, and direct observations in the field. The PPA process relies on the
data available from the project monitoring and evaluation system.
5. Methodology and process. The following methodology and process were followed in
conducting the PPA: a) review of relevant documents (see bibliography in annex
2); b) interviews with project staff and other key resource persons from the
executing and implementing agencies; c) interviews with sample project
beneficiaries; d) interviews with IFAD staff in IFAD Headquarters and in country
office; and e) interviews with sector experts as well as other multilateral agencies
involved in poverty reduction and the rural finance sector.
6. Prior to undertaking the PPA mission, key areas and data gaps were identified to be
addressed during the country visit. Additional primary data were collected in the
field to validate the findings and conclusions of the PCRV and to allow for an
independent assessment of project performance. Given time constraints, a
1 Available at: http://www.ifad.org/gbdocs/eb/102/e/EB-2011-102-R-7-Rev-1.pdf.
2 Please refer to annex 1 in this document.
2
qualitative approach was adopted for data collection. As the project continued its
activities after project completion in 2010, the two implementing financial
institutions – Chongqing Rural Commercial Bank (CRCBank) and Shaanxi Provincial
Rural Credit Cooperative (RCC) – were able to provide the PPA team with additional
data, in some cases covering activities up to 2012.
7. The PPA field mission3 in China was undertaken between 12 and 26 June, 2012.
Following a series of meetings in the capital city of Beijing, the evaluation team
visited: a) Chongqing Municipality and Youyang county, and b) Shaanxi province
(Xi’an city) and Xixian county to observe activities of the CRC Bank, the Shaanxi
Provincial Rural Credit Cooperative Union, and the Xixian Rural Cooperative Bank
(PRCCU) respectively. Two debriefing meetings were held in Chongqing and in Xi’an
city following the field visits. A wrap-up meeting was conducted in Beijing with the
Ministry of Finance to brief the Government about the preliminary findings of the
PPA mission and gather government’s views. The mission schedule and list of
persons met is attached to this report as annex 3.
II. The project
A. The project context
8. Poverty reduction and economic development. RFSP was designed and
implemented in an era when China experienced fast economic growth and rapid
poverty reduction. Contributing factors included special government programmes
to reduce rural poverty by focusing on farmers and numerous other reforms that
encouraged local initiatives. Financial sector reforms were also part of the overall
reform effort. China maintained an average annual growth of 10 per cent Gross
Domestic Product (GDP) for nearly thirty years (1980-2008). In 2009, despite the
global financial crisis, the country recorded a GDP growth rate of 8.7 per cent. GDP
per capita increased as well, from US$ equivalent 3,324 in 2008 to 3,710.72 in
2009. Rural income per capita in 2009 followed a similar trend, amounting to
US$756,68 or an 8.3 per cent increase from the previous year.4 The Government
announced the successful reduction of the number of absolute poor from 260
million in 1978 to about 14 million in 2008, based on the previous poverty line of
CNY785 as rural income per capita. In 2008, the Government increased its poverty
line standard to CNY 1,196. This in turn expanded the category officially recognized
as the poor to more than 40 million people and included also the low-income
stratum, which had been accounted for separately in previous years. The
Government of China has been drafting an action plan for the period 2011-2020 to
ensure the eradication of absolute poverty in the country. To this end, substantial
financial and other resources are expected to be allocated for this purpose. It is
important to note in this context that poverty in China also holds a geographical
dimension: central and western provinces are still considered poorer than eastern
provinces. Any future intervention by IFAD and/or other donors needs to consider
the geographical concentration of poverty, especially in remote and isolated areas.
9. Rural financial sector reform. RFSP was designed when the Government undertook
major reforms of the rural financial sector. The sector has been dominated by a
few key actors, most notably the Agricultural Bank of China (ABC), the Agricultural
Development Bank of China (ADBC), as well as thousands of rural credit
cooperatives (RCCs). All these institutions were initially regulated by the People’s
Bank of China (PBC - Chinese Central Bank). A critical institutional reform was the
establishment of the Chinese Banking Regulatory Commission (CBRC), which was
mandated, among others, to manage RCCs. Later on in 2005/06 CBRC has become
the only regulatory body of RCCs and banks instead of directly managing them.
3 Team Composition: Dewan A.H. Alamgir, Rural Finance Specialist (Consultant) and Mark Keating, Evaluation Officer,
IOE (IFAD). Mr Konstantin Atanesyan, Senior Evaluation Officer, IOE (IFAD) joined the mission on 22 June 2012 in Xi’an, Shaanxi province. Mr Sun Yinhong, IFAD Country Presence Officer and Ms. Weijing Wang, Program Officer supported the mission in planning and organization and shared their experience of the project. 4 Economist Intelligence Unit. China Country Profile, 2011.
3
10. A summary of the several changes which took place in the management of ABC
and RCCs is provided to clarify the context and performance of RFSP. RCCs
originally acted as branches of ABC. During this period, the RCC network suffered
from high share of non-performing assets, high losses, undercapitalized RCCs and
inadequately skilled staff members. These issues were related to the prevailing
policies of directed credits, controlled lending interest rate, unclear ownership, lack
of incentives to make profits and the focus on providing loans to state-owned
enterprises. The challenge of the reform implemented by CBRC was to give the
RCCs a corporate structure with clear ownership to be driven by performance
(profitability).
11. The reform process started with the release of the RCC Reform Plan by the State
Council in June 2003. The main objectives were to restructure and define
ownership and governance of the RCCs of different sizes and performance
categories and to establish a more efficient rural finance system. Since 2005, the
reform accelerated and selective RCCs were restructured into rural commercial
banks, rural cooperative banks, and rural credit cooperatives. This process
continues to date with more and more RCCUs at county level being converted to
RCBs. Also, former provincial RCCUs (RCC unions) were re-organized into regional
financial institutions, in the form of rural commercial banks, rural cooperative
banks or provincial RCCUs, depending on the market potential and asset quality of
each provincial network. County RCCUs were converted to RCBs, as sub-branches
of provincial RCBs or status quo RCCUs, and township RCCs were restructured as
service outlets of county RCCUs. By 2010 RCC organizational conversion or
restructuring was completed in most provinces. Notably, operating efficiency and
network capacity differs from one province to another, mainly due to market
conditions, capitalization of the RCCs and their differing traditions of service in the
different provinces of China. Following years of reform and change, RCC networks
remain the most extensive service providers in the rural areas: As of June 2012
China has 230 rural commercial banks, 180 rural cooperative banks and 2,265
RCCs.
12. In addition to the above, in the period 2007-08 new types of smaller rural financial
institutions were licensed: village and township banks (VTB), microcredit
companies, and rural mutual fund associations. One requirement for VTB licensing
is that it has to be initiated by a rural commercial bank. As a very successful and
one of the largest rural commercial banks, the Chongqing Rural Commercial Bank,
an RFSP partner, has already established two VTBs outside the Chongqing
province. As of mid-2012, more than 726 VTBs are in operation.
13. Impact of CBRC-led reforms:
Ownership is clear because of well-defined corporate structure and
shareholdings. Private investors have also bought shares of rural
commercial banks. Notably, the Chongqing Rural Commercial Bank
developed into a commercial bank from an RCC as consequence of the
reform process and in 2011 enlisted in the Hong Kong Stock Market;
Profitability has improved. Average return on assets and return on equity
are estimated to be 1.2 per cent and 19.2 per cent respectively;
Non-performing assets have significantly declined, and according to CBRC,
the share of non-performing loans is 1.8 per cent;
Management capacity and internal management systems were improved
and streamlined. Management information systems are computerized; and
Capitalization has improved as per new requirements.
14. Weakness in rural finance sector. Even with these different types of financial
institutions, access to finance for the poor is still low, especially in western and
central provinces. RCBs, Rural Cooperative Bank (RCopBs), RCCs have traditionally
serviced private clients that were considered non-poor as well as larger corporate
4
rural clients, including state-owned enterprises. Without outside stimulus, it is
likely that the RCC networks will continue to neglect lower income farmers, rural
households and rural micro enterprises, above all since the pressure for generating
profits has increased for the RCCs after the reforms.
15. Future direction of reform. The reform agenda is driven centrally by CBRC and the
following still needs to be considered for future reform steps: liberalization of
savings and lending interest rates; further privatization of rural commercial banks
and cooperative banks; risk categorization and loan loss provision in line with
international standards.
16. RFSP Design. The Rural Finance Sector Programme (RFSP) was designed in
2002/2003, approved by the IFAD Executive Board on 21 April 2004 and became
effective on 13 September 2005. The programme financing was estimated at a
total of US$ 21.3 million, with an IFAD loan amounting to Special Drawing
Rrights 9.95 million (US$ 14.7 million equivalent). Due to appreciation of the
Chinese currency CNY, the project cost in US$ terms has changed to US$30.4
million at completion. The programme objective was to enable poor rural people to
improve their livelihoods by increasing their access to financial services.
17. The project was designed at a time when the government and CRBC had been
undertaking major reforms of their own for the whole rural finance sector. To
reflect the reform agenda, the project design document (appraisal report) had been
substantially revised with the expectation that the project would be implemented
by CBRC and lessons from the project could be fed into the reform process.
However, that expectation did not materialize as CBRC focused on the regulatory
framework for the whole sector rather than becoming involved in the management
of RCCs, and therefore declined to host and implement the project. At the ultimate
client level, the entry rationale for the project was the significant unmet demand
among the lower and low income sections of the rural population in the two project
areas. IFAD supported the rural finance sector and government’s policy to reform
rural credit cooperatives (RCCs). The purpose of the project was to turn the credit
cooperatives into sustainable microfinance institutions for the benefit of poor rural
people and women in particular. The project was designed for a period of four
years, initially with two counties participating from each project province
(Chongqing and Shaanxi). To accelerate IFAD loan disbursements, two additional
counties joined the project in Shaanxi in 2007, as reflected in the Loan Amendment
and following the recommendation of the Mid-Term Review.
18. Initial project implementation up to MTR was very slow due to various institutional
reasons, and the Mid-Term Review in April 2007 proposed a number of changes
and implementation modalities to accelerate implementation. After MTR,
implementation progressed but disbursement was still slow as the initial completion
date of 30 September 2009 was approaching fast. Upon the request of the
Government, IFAD extended RFSP’s completion to 31 March 2010 and closing to 30
September 2010.
B. Project implementation
19. Implementation Approach. In the above context, RFSP adopted a strategy of
working in concert with the CBRC to contribute to policy reforms, experiment with
the impact of these reforms in two provinces, strengthen institutional and
operational capacities of the two provincial RCC networks, and enhance access to
finance for the poor, women and farmers. To achieve these objectives, the focus
was on providing technical assistance in policy areas, capacity building assistance
to upgrade systems and human resources and lines of credit to expand outreach of
services. Credit lines were provided specifically to help catalyze innovative
microfinance practices and to demonstrate the operational viability of well-designed
credit products. But, as mentioned earlier, CBRC eventual non-participation forced
5
the project to work directly with the two provincial RCC networks. This affected
outcomes of Component 1 of the project.
20. Components at appraisal. The Appraisal Report included the following five
components, to which some adjustments were made at the time of the MTR:
Component 1: Policy development. key outputs and activities were:
(a) Lending rate liberalization;
(b) Microfinance group lending, women window;
(c) New product development;
(d) Technical and consultancy support; and
(e) Policy support and technical advice at national level.
Component 2: Institutional development. Key outputs and activities
were:
(a) Infrastructure and equipment;
(b) Staff training; and
(c) Reorganizing legal structure of RCCs.
Component 3: Operational development. Key outputs and activities
were:
(a) Accounting system and loan policies; and
(b) Staff incentive system.
Component 4: Financing (Line of credit). A credit line (re-financing)
was provided to help test innovative products, and to promote loans to the
poor and women.
Component 5: Programme management. Unusually, two parallel Project
Management Offices (PMOs) were established at provincial level with County
PMOs in each county covered by RFSP. The PMOs were to ensure
programme implementation. A National Coordination and Management
Office (NCMO) was proposed to maintain the overall supervision of the pilot
activities in the programme provinces. It was also to advise on national
policies and policy reform, coordinating technical assistance for studies and
staff training of the programme RCCUs. The NCMO including CBRC was
never constituted and the two provincial PMOs had to take the lead in
programme implementation.
Review of performance
1. Component 1- Policy development. The RFSP differs from other IFAD-supported
interventions, which are area-based poverty reduction projects, in the past often
containing lines of credit for pro-poor RCC outreach. In RFSP, IFAD was aiming at
helping to institute systemic, sector-wide change through influencing on-going
reform in the sector under the leading role of CBRC. This did not materialize, and
the pivotal Component 1 has become the least successful of all components. The
PCR does not clearly describe the reasons behind such situation. The following
paragraphs briefly present an analysis of the related issues and factors that led to
such situation to benefit future IFAD project design and implementation.
2. At the time of design, the Ministry of Agriculture was the proposed executing
ministry for the project, but later handed over the project to the Ministry of
Finance. This created a gap regarding the understanding about the purpose,
process of implementation as well as scope of the project. Representatives from
both the Ministry of Finance and the CBRC had associated with IFAD design
missions, but their participation did not percolate up to their respective institutions.
6
Consequently, no ownership for RFSP was generated out of their mission
participation suggesting that the participation might have been on individual rather
than institutional capacity, especially in the case of CBRC. The design document
repeatedly mentions CBRC as the executing agency but in practice it never
materialized. The project struggled in terms of achieving the objective of this
component, of day to day supervision and could not provide feedback on lessons
learned and experience gained to CBRC to bring about systemic changes in the
sector. Discussions with IFAD staff and consultants provide two probable reasons:
a) consultations/discussions of the design team with CBRC at the time of design
missions did not lead to formal approval of the project within CBRC’s internal
official system as the host and main implementer; and b) CBRC had been
managing the networks of RCCs but later, as a result of institutional reform by the
government, turned from managing RCCs into playing a regulator role. CBRC
therefore kept away from directly managing a reform project that included
activities of two RCCs and a line of credit. This decision has practically de-linked
the project from CBRC although it officially remained as the executing agency.
Therefore, RFSP could not contribute to the sector reform process without any
formal mechanism for interacting with CBRC in the context of RFSP.
3. There was no adequate discussion and understanding with the two RCCUs about
their role in policy reform. These networks operate under policies of CBRC and are
not in a position to influence CBRC policy reform agenda.
4. The MTR recommended the formation of a policy consultative mechanism under the
project framework, comprising one representative from central CBRC, one senior
manager, two senior professionals and one support staff from each program
PRCCU. There was no follow up on this proposal as well. However, eight sessions of
policy consultation between the two project RCC networks were organized, mostly
to regularly share experiences in rural lending and programme implementation
management.
5. New product development. The two implementing agencies made significant
progress of their own in product development after MTR guidance from 2007
onwards. The PCR reports that the two provinces developed 29 new loan products,
of which 26 were applied and rolled out through the provincial networks.
Meanwhile, other products such as savings products and intermediary services
were designed and tested in the rural market and 45 other new products were
reportedly applied in the market, of which four are related to savings.
6. In the course of the PPA mission, the CRC Bank reported developing 35 new loan
products, of which 17 are related to agriculture. In particular, four products were in
high demand: a) loan for returnee migrant workers to start business; b)
management of forestry products; c) joint loan; and d) production loan for women.
All these products were not only introduced in the two project counties (Youyang
and Younyang) but also in all other counties of the province through the rural
branches. Similarly, the PRCCU in Shaanxi also reported developing and rolling out
loan products in the four project counties covered in this province. Important to
note is that the two organizations could develop and roll out financial products of
their own. CRC Bank reported that its ‘management of forestry’ product has been
promoted by CBRC in other provinces and it enjoys strong support from the
regulator.
7. Loan pricing. This activity area was considered particularly relevant at appraisal.
The Appraisal Report states that interest rates charged by RCCs for their loans are
below market clearing levels, and their main effect is the provision of relatively
cheap resources to the State enterprise sector, the banking sector’s main client.
There was no progress made on this, as the interest rates levels at RFSP start up
and loan closing demonstrated. The Shaanxi RCCU and Chongqing RCB adopted a
more flexible pricing policy and decentralized pricing decision-making at county
7
sub-branch level since 2007. Some training sessions were reported as programme
activities to support the introduction of such decentralized pricing; there was no
substantial fluctuation in lending interest rates in the rural areas afterwards as
CBRC has provided a band for variation of interest rate for rural commercial banks,
rural cooperative banks, and RCCs. Such stable rural lending rate typically ranged
from 7 to 11 per cent for on-farm financing, and up to 15 per cent for trade and
off-farm income-generating activities. Both agencies followed the regulator’s
guidelines in this respect. See table 1 for interest rates determined by CBRC over
the years.
Table 1 RCB and RCC savings and lending rates as determined by the central bank
Year Central bank base rate,
savings Central bank base rate,
lending Rcb & rcc av. Applied
lending rates
2007 3.87% 7.29% 14.58%
2008 2.52% 5.58% 11.16%
2009 2.25% 5.31% 12.21%
2010 2.50% 5.56% 12.79%
2011 2.75% 5.81% 13.36%
Source: Project Completion Report (2011)
8. Overall assessment of the component. The main objective of the component to
influence systemic policy change was not achieved due to inappropriate
institutional arrangements; yet, the two implementing agencies developed and
rolled out financial products to suit the demand of the small borrowers. No
progress was made on making interest rates for RCC deposits and loans more
market oriented. The reform agenda has been driven nationally and centrally by
the CBRC without any link to the project.
9. Component 2 - Institutional development. Of the three designated activities –
infrastructure and equipment, staff training, and support reorganization of legal
structure of RCC- under this component the first two were successfully completed.
A total of 78 service outlets in two provinces were rehabilitated by renovating office
building and upgrading the computer and surveillance system. The PPA mission
visited a few RCC outlets in Youyang county (Kiampo market). For distant clients
CRC Bank has outstations (a cash dispensing machine and a person) located in
shops. All these outlets and branches enjoy computerized accounting and
management information systems that help provide prompt services to the clients
and improved internal control and management of the branches. The local bank
officials reported that, due to physical upgrading, the number of clients is
increasing.
10. Staff development training. A series of topics for training were identified, mainly
focusing on accounting and financial management, credit management, and
microfinance. Over the project period, 5,860 staff members (3,260 in Chongqing,
of which 27 per cent women, and 2,600 in Shaanxi PRCCU, with 15 per cent
women) have received training. It is important to note that both organizations
designed and developed training materials internally and resorted to RFSP financial
assistance only when necessary. The PPA mission had an opportunity to meet
several officials who received such training. All of them were positive about the
contents of the courses, and utilized the knowledge acquired in day-to-day
business, for example, in selecting clients, disbursement and collection of loans,
and maintenance of accounts.
11. Reorganizing legal structure of RCCs. The initiative for legal restructuring was
undertaken by the RCCs themselves under the reform agenda of CBRC and
therefore this activity was not implemented by the project. The MTR re-allocated
8
the resources initially allotted for legal restructuring. In Chongqing, former county
RCCUs were converted to sub-branches of the Chongqing Rural Commercial Bank.
Interestingly, the bank has been so successful that it had enlisted in the Hong
Kong stock market in 2011. The PRCCU in Shaanxi now has four rural commercial
banks, 12 cooperative banks and 98 RCCs as members, all of which have separate
legal identity. The success of such transformation shows the effectiveness of the
reform engineered by CBRC.
12. Overall assessment of the component. Only two of the three activities have
been completed successfully; one activity was beyond the rim of the project and
was readjusted during MTR.
13. Component 3 - Operational development. This component dealt with making
accounting principles and practices compatible with internationally-accepted
accounting standards. The significant exception is classification of impaired assets
and loan loss provisioning. During the project implementation period, there were
even developments away from a strict time based loan loss provisioning. Risk
categorization for loan portfolio at risk and attendant loan loss provisioning are less
in line with international standards in the RCCs than they were a few years ago.
14. At RCC level, loan polices and loan management needed to be revised to match
demand for new loan products. Notably in the area of staff incentive systems, no
progress seems to have been made. The impact of revised loan policies can be
seen in the development of new loan products. The accounting system and
presentation of financial reports conforms with national standards.
15. Operating self-sufficiency. The impact of the overall reorganization of RCCs can
be seen in improved operations. Both provincial networks reported improved
operating self-sufficiency,5 for Chongqing from 120 per cent in 2005 to 126 per
cent in 2009, and for Shaanxi from 115 per cent to 152 per cent during the same
period. The improvement in operating efficiency derives from the improved
productivity of staff members. Staff productivity has improved in the programme
counties since the start of the RFSP. In 2005, an average 160 to 250 active
borrowers per credit officer were reported, while in 2009 programme lending
recorded 631 active borrowers per credit officer in Shaanxi and 807 in Chongqing.
The main contributor behind improved operating efficiency is the drive to make
profit by reducing cost. This high efficiency was also possible due to the fact that
clients repay mostly in one instalment, that is, at the end of loan period and a
credit officer need not to visit clients too frequently.
16. Overall assessment of the Component. One activity has been completed
successfully with overall adequate impact on financial and operational performance
of the two implementing agencies.
17. Component 4 – Financing (Line of Credit). Under Component 4, the project
provided a line of credit to the two implementing agencies to expand loans to rural
low income clients and to try out microcredit group lending techniques. The credit
reimbursed on-lending funds for pro-poor lending through the two institutions. The
microcredit lending under the project was executed mostly through the RCC
operations of individual microcredit lending, with more conscious targeting of the
poor and the women as differentiated from the RCC conventional microcredit
portfolio. In 2007, the MTR proposed a village credit model to be experimented in
the programme counties. The PPA mission found no evidence of the application of
such village credit model and both rationale and use for the IFAD targeted clients is
not discernible.
18. Clients and client selection. The two institutions follow systematic procedures to
identify potential clients and assess risk. While discussing with sample clients, it
5 A percentage which indicates whether or not enough revenue has been earned to cover the total costs – operational
expenses, loan loss provisions and financial costs.
9
was found that they either learned about the financial services offered by the CRC
Bank and PRCCU from their neighbours, from visiting credit officers or from market
campaign organized by them. This indicates systematic efforts to reach rural and
women clients. The credit analysis includes indicators such land ownership, total
asset, purpose of the loan, reputation of the applicants and the loan size is
determined accordingly. For poor applicants loans start with smaller amounts but
gradually increase. The PCR reports that 36,224 clients were given loans during the
project period; of these, 47 per cent was classified as poor and low income. Twenty
eight per cent of total beneficiaries were women. The sample of clients interviewed
by the PPA mission seem to be satisfied with the services provided; loans were
disbursed by RCCs within a week of application, and they plan to access further
loans in future.
19. Loan disbursement and recovery. Most of the loans are only for a 12 month
duration and probably not well attuned to client’s requirements for term financing.
Only loans for house construction could be up to three years. The absence of term
lending facilities for small scale investment finance is all the more surprising when
the unusually long terms of RCC term deposits are considered. Loans are recovered
with interest and principal amount at maturity; such arrangement is the preferred
one by the clients, although can increase risk if clients miss repayment dates. Term
lending for small scale investments and repayments in instalments should be
preferred depending on the purpose of the loan.
20. Loan range. Loans up to CNY 70,000 (US$ 11,000) have been disbursed for house
construction. Most of the loans reportedly range between CNY 10,000 to 30,000.
CRC Bank has the following policies regarding loan range, as illustrated in table 2.
Table 2 CRC Bank: Loan products, terms and conditions
Name of the loan products
Duration of the loan (in years) Range of loan amount (CNY)
1 Loan for migrant workers (business start-up)
1 to 3 20,000-200,0000
2 Start-up loan for women 1 to 3 10,000-50,000
3 Forestry ownership management loan 1 to 5 20,000-200,0000
4 Joint loan 1 to 3 10,000-50,000
Source: CRCBank
21. Interest rate. Interest rates are within the range stipulated by CBRC which, in the
case of CRCBank and PRCCU vary between 7 to 15 per cent. As noted earlier, the
rates are determined by the regulator, and not determined by cost or market
considerations.
22. Use of loans. In line with economic conditions in the six project counties, most of
the loans - except those provided for house construction - are directly or indirectly
linked with agricultural production. Table 3 below provides aggregate CRCBank
loan information for 2011.
10
Table 3 Microfinance portfolio of CRCBank in 2011
Loan category Households Amount (million CNY) Percentage (%)
Crop growing 48,937 880.73 41.9%
Livestock raising 21,793 481.14 22.9%
Commercial and trade 18,292 365.84 17.4%
Miscellaneous 16,652 373.75 17.78%
Total 105,674 2101.46 100%
Source: CRCBank
23. It is important is to note that the two financial institutions are continuing their
microfinance product related activities beyond the project period. Table 4 below
provides information on the microcredit portfolio of CRC Bank in two project
counties for the 2010-2012 period.
Table 4 CRC Bank RFSP-Microcredit information beyond project period in two project countries
Year and county
Loan disbursement during the year
[Million CNY] Number of borrowers
Man Women Total Man Women Total
2010 Youyang 16.55 1.81 18.36 1,341 185 1,526
Yunyang 15.96 1.91 17.87 1,004 205 1,209
Total 32.51 3.72 36.23 2,345 390 2,735
2011 Youyang 17.42 2.59 20.01 1,024 249 1,273
Yunyang 16.41 2.75 19.16 902 237 1,139
Total 32.51 3.72 36.23 1,926 486 2,412
2012
(Jan.-Apr.)
Youyang 12.36 1.57 13.93 689 153 842
Yunyang 12.76 1.63 14.39 612 178 790
Total 25.12 3.2 28.32 1,301 331 1,632
Source: CRCBank
24. After more than four years of implementation, PMOs in Chongqing and Shaanxi
accounted 36,224 direct beneficiary households from their programme microcredit
lending, of which 10,246, or 28 per cent, were women. Indirect beneficiary
households were estimated to be approximately 830,584, which is equivalent to
the number of savings households after deducting the overlapping rate of 15 per
cent. However, the PPA mission found no supporting evidence to the claim that
each and every depositing household in the counties covered in the two provinces
is either a direct or indirect beneficiary of RCC project loans. In addition, the PPA
mission questions the comparatively low number of 36,224 direct beneficiary
households in relation to the number of indirect beneficiaries as high as 830,584.
25. Overall assessment of the component. The component has been successfully
implemented, namely expanding access to credit for the poor, women and farmers,
both by using project credit lines as well as the own resources of the two RCC
networks, which even surpassed the project credit funds. However, the high
amount of financing coming forward from the partner institutions puts a question
mark behind the very rationale of operating a credit line, a point further
accentuated in the most recent IFAD Rural Finance Policy of 2009.
11
26. Component 5 Project Management. The PMOs in the two provinces completed the
Results and Impact Management System and benchmark surveys twice during the
programme period, respectively in 2007 and in 2010. A number of management-
related training sessions were reportedly organized and conducted for managers
and staffs. Female participation in those training sessions was estimated at 15 per
cent of the total.
27. Overall assessment of the component. PMOs managed well all activities that fell
within their capacity, especially after implementing the recommendations contained
in the MTR. They were not in a position to implement most of the activities under
Component 1 due to the reasons previously described.
III. Review of findings
A. Project performance
Relevance
28. At the time of design in 2002/3, the RFSP was relevant in terms of identifying the
general direction of reform agenda for networks of RCCs as well as strengthening
capacity of two partner RCCUs while they were undergoing restructuring. The
concept of designing a project to assist the Ministry of Finance or the Chinese
Banking Regulatory Commission (CBRC) remains relevant to date to implement
future reforms. At the same time, the project was designed and implemented at a
time when the Government was carrying out a major reform of the rural finance
sector. Some of the stated objectives of the RFSP equaled the objectives of the
reform program: increasing access to credit, improving the profitability of rural
credit cooperatives (RCCs), and promoting institutional reform of the RCCs. The
RFSP had a policy component which was designed to assist in ongoing and planned
policy reforms. However, the institution responsible for policy reforms (the China
Banking Regulatory Commission, CBRC) did not agree to be the implementing
agency for the project, and thus the reform process proceeded “in parallel” with
the project.
29. Up to the 2007 MTR, the progress was slow reportedly due to procurement related
issues, and delays in initial mobilization. The MTR recommended reallocation of
resources to speed up IFAD loan disbursements; notably, despite changes in the
priority areas of investments, the RFSP remained relevant to its overall objectives.
30. Under Components 2 and 3, the project assisted to upgrade management policies,
institutional and operational capacities, and sustainable lending practices to target
segments. The project has been supportive of organizational conversion in
Chongqing during 2007/08 from Rural Credit Cooperative Union to Rural
Commercial Bank (CRCBank). The project’s assistance helped the newly formed
Rural Commercial Bank to keep some focus on providing financial services to the
poor, women and farming while the bulk of its loans had been disbursed to larger
businesses; this was possible due to the project’s priorities in management,
operations and rural targeting. In Shaanxi, according to the provincial and county
level officials, the project helped upgrading systems, developing financial products,
and keeping the focus on the poor and farmers. The products developed for the
project counties have been replicated in other counties.
31. The project proved to be directly relevant to the poor, women and small borrowers.
Both partner financial institutions extend financial services (savings and credit) to
the poor, women and farmers. The RFSP paid particular attention to product
development, by trying to promote microcredit products with poverty and gender
sensitivity.
32. The selection of the RCC network as implementing financial institutions has been
appropriate and will remain so for the future as well. The RCC network is the only
practical way of reaching large rural areas with financial services. They are rooted
in the rural area, and had a long history of supporting agricultural and rural
12
development. Any future rural finance project, if undertaken in poorer provinces in
the central and western provinces has to be partnered with respective RCCs.
Whether net asset increases for poor productive people can be achieved without
lines of credit as in the most recent additions to the IFAD investment portfolio
remains to be seen.
33. Rating of relevance. The PPA finds the project concept relevant both at the time of
design as well as at present, although the design process lacked full
comprehension of the reform process, and proper institutional arrangements for
influencing systemic changes which led to some extent to partial underperformance
of the project. Relevance is therefore rated as moderately satisfactory (4).
Effectiveness
34. The effectiveness of a project is defined as ‘the extent to which the development
intervention’s objectives were achieved taking into account their relative
importance’. Since there was no logframe at the time of design, the project later on
produced a logframe at MTR (see annex 4) that does neither provide a statement
by component, nor any physical target for measuring progress. The PPA mission
has first used the overall objective of “enabling poor rural people to improve their
livelihoods by increasing their access to financial services”, and the overall project
purpose “to turn the credit cooperatives into sustainable microfinance institutions
for the benefit of poor rural people and women in particular” as a reference point to
assess the effectiveness of the project.
35. Components. As per resource allocation, the line of credit was the most important
component to actually increase access to finance by providing matching capital to
the two financial institutions. Policy reforms, institutional and operational
development activities – with the exception of infrastructure (civil works and
computer hardware) did not require large sums of money but contributed to overall
capacity building of the institutions and the sustainability of the financial services.
Resource allocation as a percentage of total costs was as follows: (1) Policy
Development (4.65%); (2) Institutional Development (12.85%); (3) Operational
Development (2.62%); (4) Financing (line of credit for microfinance) (76.62%);
and (5) Management (3.26%).
36. Effectiveness in relation to the overall objective. The project contributed to
improving livelihoods of the poor people by increasing access to financing services.
The two RIMS studies clearly show that borrower families have higher income and
increased assets. It should be noted that increased income cannot be fully
attributed to access to credit (attribution problem) as exogenous factors may have
influenced higher levels of incomes as well as increased assets.
37. Effectiveness in relations to overall purpose. The overall project purpose ‘to
turn the credit cooperatives into sustainable microfinance institutions for the
benefit of poor rural people and women in particular’ was rather ambitious.
Reforms of the RCC networks were carried out which led to greater clarity in the
corporate structure and ownership, improved profitability, a significant decline in
non-performing assets, and improved capitalization. Participating financial
institutions trained staff and developed new financial products suitable for the poor,
women, and farmers. However, because the project was de-linked from CBRC, and
because Participating financial institutions trained staff and developed new financial
products on their own, it is not clear to what extent these outcomes can be
attributed to the project. However, the project can reasonably take credit for
organizing a line of credit for microfinance related activities, assistance in
infrastructure development and human resources development. Both institutions
are financially sustainable and their overall profitability will allow and encourage
them to continue with microfinance operations as a part of their portfolio.
13
Effectiveness by project components
Component 1 - Policy development. This component partially met its
objectives. The only effective activities carried out were inter-provincial
exchanges between the two programme networks, and some training and
internal consultations in lending policy development. The new product
development has been carried out by the two financial institutions.
Component 2 - Institutional development. This component was effective as
the project contributed to infrastructural development such as renovation of
outlets, computerization of internal systems and human resources development
through a large number of training sessions. As a result, it contributed to
attracting clients, enhanced competitiveness of the institutions and enhanced
internal capacity, and made the credit officers of RCCs effective in targeting
poorer clients.
Component 3 - Operational development. This component was partially
effective as it contributed to revising internal policies, yet the review of the staff
incentive system could not be achieved.
Component 4 - Line of credit. The most effective component of the project
was finance for microcredit. Both the IFAD loan and the matching funds from the
two financial institutions were used. A number of reasons made this component
effective, among these the fact that it was not subject to any form of
procurement, and that line of credit was the main business of the two financial
institutions already prior to project start up. Their capacity to develop and
promote successful new loan products was another important factor to be
considered.
38. Regarding the review of the financial achievements, table 5 indicates that the line
of credit component achieved 100 per cent of the implementation target, as it
adopted the RCC’s own methodologies. However, the requirements for accessing
IFAD finance were perceived as burdensome when it came to procurement for
consultancy services and outsourced training.
Table 5 Financial achievements, overall programme (in US$ equivalent)
Overall programme targets Overall programme achievements
Overall percentage of programme achieved
IFAD RCC Total IFAD RCC Total IFAD RCC Total
Components Policy dev. 2,130,325 36,069 2,166,394 1,344,959 70,491 1,415,449 63% 195% 65%
Institutional dev. 4,249,329 358,324 4,607,653 3,626,424 281,590 3,908,014 85% 79% 85%
Operational dev. 853,624 18,416 872,040 757,960 39,893 797,852 89% 217% 91%
Financing 9,014,740 13,115,311 22,130,051 9,394,080 13,915,261 23,309,341 104% 106% 105%
Management 1,388,962 89,764 1,478,727 764,980 226,366 991,346 55% 252% 67%
Total 17,636,981 13,617,884 31,254,865 15,888,403 14,533,600 30,422,003 90% 107% 97%
Categories
Civil work 985,619 51,875 1,037,494 931,103 49,005 980,108 94% 94% 94%
Equipment & materials
1,712,203 199,513 1,912,316 1,327,827 147,536 1,475,363 78% 74% 77%
vehicles 209,511 43,214 252,725 196,793 30,925 227,717 94% 72% 90%
Studies 660,317 14,530 674,847 322,204 16,388 338,592 49% 113% 50%
Training & consultancies
4,281,885 134,906 4,416,791 3,107,609 160,091 3,267,700 73% 119% 74%
incremental credit
9,014,740 13,115,311 22,130,051 9,394,080 13,915,261 23,309,341 104% 106% 105%
Staff salaries 238,249 28,974 267,223 318,746 199,128 517,874 134% 687% 194%
Operating costs 534,456 29,562 564,018 290,042 15,265 305,307 54% 52% 54%
Total 17,636,980 13,617,885 31,255,464 15,888,403 14,533,600 30,422,003 90% 107% 97%
Source: Project Completion Report (2011)
14
39. Overall effectiveness. The project could be considered effective judged by impact
on livelihoods and expanded access to credit. However, in terms of policy reform
and its contribution to transforming RCCs into sustainable institutions, the extent of
the project contribution is not fully clear. At the same time, given the successful
achievements with regards to component 4, the criterion of effectiveness is
considered to be satisfactory (5).
Efficiency 40. Efficiency is a measure of economical use of funds, expertise and time of the
project to achieve objectives. As discussed earlier the efficiency was affected due
to delay in implementation. As noted in the MTR, not much progress was made
before 2007. Besides, significant amounts of time were lost between loan approval
and effectiveness - 16.8 months, which is well above the regional average of 9.2
months and the IFAD average of 12.4 months. The implementation period was
extended by six months, bringing the current completion date at 31 March 2010
and the current closing date at 30 September 2010.
41. Fund disbursement. Due to the appreciation of the Chinese currency and the
Special Drawing Rights against the US Dollar during implementation, the project
made more financial progress than physical progress. Table 6 below presents
revised estimates and actual financial performance. The revised project estimates
at the time of the MTR amounted to US$ 31.25 million while actual achievement
was US$30.422 million, that is, 97 per cent fund utilization. The IFAD loan figure
was revised to US$17.64 million against the original figure of US$14.7 million at
the time of appraisal. Against the revised estimates, actual IFAD loan disbursement
was US$15.89 million, which is 90 per cent of adjusted financial value. Similarly,
RCCs have contributed US$14.533 million against a revised estimate of US$13.617
million (107 per cent), that is, the RCCs financial contribution has even exceeded
the revised estimate. RCCs contribution even look much higher if compared to
appraisal estimates. At appraisal, the IFAD loan was estimated to be US$14.7
million, that is, the actual disbursement was 8 per cent higher whereas RCC’s
contribution was estimated at US$6.6 million, and the actual disbursement was
120 per cent higher. Therefore, the IFAD contribution to the total financing was
weighted for 52 per cent instead of 70 per cent at appraisal, and the final
counterpart finance averaged at 48 per cent as compared to the initial figure of
30 per cent. During implementation, both provincial RCCUs/RCBs provided
continued financial support through matching funds to support programme
implementation.
Table 6 Programme costs, financing and matching in US$
IFAD RCC Total
Adjusted cost and achievement
Adjusted programme target 17 636 980 13 617 885 31 255 464
Actual realizations 15 888 403 14 533 600 30 422 003
Percentage of actual realizations 90% 107% 97%
Consolidated programme matching percentage 52% 48% 100%
Province-wise matching in Shaanxi 43% 57% 100%
Province-wise matching in Chongqing 66% 34% 100%
At appraisal
Appraisal estimation 14 700 000 6 600 000 21 300 000
Changes in percentage* 8% 120% 43%
Source: Project Completion Report (2011)
* Total change is caused by the valuation of Chinese currency; increased counterpart percentage is caused by both currency valuation and RCC additional funding.
15
42. The MTR reallocated funds due to procurement difficulties expressed by both
provinces in executing a number of activities related to consultancies, studies and
policy development. The line of credit was increased but allocations for training and
consultancies at provincial levels reduced. The actual project cost by component is
given below in table 7. This reallocation has contributed to a more efficient use of
funds as credit disbursement component has exceeded appraisal targets and
directly contributed to achieving the project objective and purpose. However, the
contribution of the credit line should also be seen in comparison with total assets of
the two financial institutions. It is estimated, as of 2009, the project credit fund
amounted to only 3 per cent of total assets. This small size prohibits rigorous
efficiency analysis. The project acted as catalyst, or influencing agent, to develop
new financial products for the poor and encourage the financial institutions to keep
in their portfolio.
Table 7 Actual fund use by component (in US$ equivalent)
All programme achievements
Components IFAD Gov./RCC Total % of Total
Policy dev. 1 344 959 70 491 1 415 449 4.65
Institutional dev. 3 626 424 281 590 3 908 014 12.85
Operational dev. 757 960 39 893 797 852 2.62
Financing 9 394 080 13 915 261 23 309 341 76.62
Management 764 980 226 366 991 346 3.26
Actual total 15 888 403 14 533 600 30 422 003 100.00
Source: Project Completion Report (2011)
43. The management cost of the project is very low (3.26 per cent of total cost), which
allowed more than 96 per cent of resources to be allocated for program activities.
But the probable reason for such low management cost is that the microcredit
management cost has been absorbed by the financial institutions as their regular
operating costs. In any case, partnership with appropriate institutions has made
the project more efficient.
44. The PCRV compares RFSP with two other IFAD-funded projects in China which were
on-going in the same period and having a rural finance support component. As
shown by data in table 8 below, RFSP did reach a much smaller number of direct
beneficiaries. The main reason probably lies in loan size (amount disbursed per
loan per person), that is, RFSP disbursed larger loans per person than the other
two projects and the beneficiaries targeted by the other two projects are probably
much poorer that the ones targeted by RFSP.
Table 8 Comparison between IFAD projects
Project RF Costs
CNY million Total project cost
CNY million Proportion
investment in RF Borrowers
reached by RF
Qinling Mountain Area Poverty- Alleviation Project 129 695.64 19% 459 300
West Guanxi Poverty-Alleviation Project 101 730 14% 239 000
Rural Finance Support Programme 126 126 100% 36 224
Source: IOE interim evaluation Qinling mountain area poverty-alleviation project
16
45. Overall efficiency. Taking into account the financial progress against the physical
progress and the slow start-up, this PPA rates efficiency as moderately satisfactory
(4).
B. Rural poverty impact
46. Data sources and limitations. The following analysis is based on data presented
in the project completion report and complemented by additional information from
the PPA field visits. The project conducted two RIMS surveys in mid-2007 and early
2010. However, no baseline data and comparison with control groups are available
to attribute that the impacts are directly correlated with the project interventions,
especially enhanced access to financial services. A random selection of 900
households in the project villages from the two provinces (180 households in
Chongqing and 780 households in Shaanxi) was reviewed. The results represent
overall changes in the households of the selected villages but do not attribute the
changes to the project interventions. A more appropriate methodology would have
been to compare changes in beneficiary households with control groups to attribute
the changes to project interventions.
Household income and assets
47. No specific data were collected regarding household income and income generating
activities. The PCR noted an overall reduction in poverty in project villages. For
example, in Chongqing, poor or low-income households decreased from 16 per
cent in 2007 to less than 5 per cent in 2009; in Shaanxi, surveys recorded 11 per
cent of poor and low-income people in 2007, which was reduced to about 8 per
cent in 2009. However, it also noted that the improvements correspond with the
overall poverty reduction in the country. That makes it difficult to segregate the
impacts of the project on income and poverty reduction. The PPA mission
interviewed a small number of direct beneficiaries. The interviews indicate that the
microcredit recipients could expand business (number of livestock) and buy inputs
for farming (for example, tobacco and spice production) that led to increase in
production. At the same time beneficiaries get good farm gate returns due to
better communication and access to markets. Besides, families are increasing
income from other sources as well as, for example, part-time or seasonal wage
labour outside the villages. Therefore, a combination of factors, where credit is one
element, led to increased income.
48. Other indirect or proxy indicators mentioned in the PCR may be considered to have
contributed to increased income. Through its support to the RCC/RCB networks,
RFSP contributed to an increased access to financial services. According to the
RIMS data, access to RCC loans by poor rural households increased from 36 per
cent in 2007 to 42 per cent in 2009 for Chongqing and from 37 per cent to nearly
60 per cent in Shaanxi in the same period. Additional access to finance usually
leads to expansion of business and good market prices lead to additional income.
This link was corroborated by interviews during the PPA mission.
49. Access to savings services also indicates increase in financial assets. The
percentage of households having a savings accounts increased dramatically from
57 per cent to 90 per cent in Chongqing, and from 49 per cent to 70 per cent in
Shaanxi. This could indicate more net savings. In reality however, the opening of a
savings account is a pre-condition for receipt of government subsidies for small
farmers, and this may be the reason for more savings accounts opened over the
RFSP implementation period. Without knowledge on minimum and average
balances kept in these accounts over the past years, it is more likely that subsidy
receipt rather than genuine savings accumulation was the prime driver for account
openings in RCCs.
50. As also noted during the PPA mission, the two RIMS surveys reported strong and
increasing involvement of farmers in income generating activities related to
marketing, transport and trading. This change is a natural progression in a rapidly
17
developing rural economy where farmers are increasingly producing diversified as
well as high-value products demanded by the markets and venturing into other
non-farm income generating activities. In Chongqing, 83 per cent of farmers
reported such involvement and practice in 2007 and 98 per cent in 2009; similar
trends were observed for Shaanxi, with 84 per cent and 97 per cent of farmers in
2007 in 2009, respectively reporting these positive developments. Yet, all these
changes cannot be attributed only to credit as mentioned above but to a result of
many factors.
51. The PCR tried to indicate increase in income by measuring changes in household
assets. The RIMS surveys show an increase in household assets such as having a
TV and a motorcycle. Positive changes in farm tools can be observed moving from
mainly hand tools in 2007 to animal-driven or machine powered tools in Shaanxi,
and animal-driven tools in Chongqing. The emphasis on animal driven tools in
Chongqing was explained by the small plots in the predominantly mountainous
area. With regard to livestock, the 2009 survey observed that more households
were able to raise larger animals (cattle and pig ownership increased from 22 per
cent to 61 per cent in Chongqing). The PPA mission found that beneficiaries
borrowed loans to repair or build houses, which also reflects improvement in assets
and standard of living. However, there is no evidence that all such changes can be
attributed to credit only.
52. Overall, the changes reported here are typical of microcredit programs elsewhere.
The PPA rates the project impact for household income and net assets as
satisfactory (5).
Human and social capital and empowerment
53. Direct beneficiaries. The project did not undertake any training at client level. All
such training was directed to staff members of the two financial institutions; yet,
clients did indirectly benefit from the credit program that improves social capital
and leads to empowerment in the long run. The PPA mission learned that many of
the clients were first time borrowers, both men and women, - the mission met a
few - who learned about the financial services provided by the RCCU/CRCB and for
the first time interacted with formal financial institutions. Several women opened
savings accounts and also used loans, although men are still ahead in using
financial services. This gradual process leads to long-term empowerment.
54. RFSP has substantially invested in RCC staff training. In Chongqing, 3,260 staff
were reported being trained of which 27 per cent were female participants; in
Shaanxi, 2,600 staff received training of which 15 per cent were female. Training
areas ranged, among others, from integrated operational networking to microcredit
lending, risk management, new loan product piloting, small and medium enterprise
lending, loan categorization, analysis of operational efficiency and financial
sufficiency. Those actions were part of the RCCU/RCB own capacity building but
were strengthened through programme support and they have helped improving
the human assets of the RCC/RCB networks in the two programme provinces.
55. Conceptual introduction and exercise of market segmentation, lending with poverty
and gender sensitivity or with differentiated services have contributed to the
empowerment of the vulnerable groups such as poor, women and returning
migrant workers. According to the Completion Report, their socio-economic status
tends to improve. For example, the surveys indicate an increasing trend of women
signing RCC loan contracts and, especially, becoming savings account holders. For
the latter, an increase of 40 per cent was noted for female savings account holders
in Chongqing.
56. Based on the above analysis, programme impact on human and social capital and
empowerment is rated as moderately satisfactory (4).
18
Agricultural productivity and food security
57. The RIMS survey attempted to capture changes in agricultural productivity
irrespective of sources of interventions. The Completion Report highlights an
increased percentage of households facing increased yields in their agriculture
production. Therefore, the question remains on what are the reasons for such
positive change. The Completion Report explains the improvements as results from
a combined support of government programmes and a more diversified market for
local farmers. Yet, the project may take credit for having enhanced clients’ ability
to grasp opportunities and venture out in agricultural as well as other income
generating activities.
58. The completion report highlights findings of the 2007 and 2009 surveys in terms of
food security, child malnutrition and safe sanitation. According to the survey results
of 2007, approximately 2 per cent of households in Chongqing had faced food
shortages. There was no report of households experiencing food shortages in 2009.
Overall, children’s nutritional status improved between 2007 and 2009. For
example, stunting decreased significantly in Chongqing, from 52 per cent to 35 per
cent. In Shaanxi however, it did decrease only slightly. The presence of chronic
malnutrition in the two provinces seemed to be caused by local food structure,
where nutritive balance may need to be re-assessed. Access to safe water
increased significantly in both provinces, to almost 100 per cent. However, safe
sanitation did decline in Shaanxi from 53 per cent in 2007 to 50 per cent in 2009
because of the long-lasting drought in the northern project counties where water
use was reduced to cover drinking and irrigation needs first.
59. Notably. Food security has improved and malnutrition has been reduced. This
positive result can be rated as satisfactory (5).
Natural resources and the environment
60. Similar to agricultural productivity it is difficult to measure any direct impact of the
project on natural resources and environment. Given the lack of data and evidence,
this criterion has not been rated.
Institutions and policies
61. There were a number of accomplishments in the area of policy and institutional
reform. Reforms of the RCC networks were carried out which led to greater clarity
in the corporate structure and ownership, improved profitability, a significant
decline in non-performing assets, and improved capitalization. However, because
the project was de-linked from CBRC, it is unclear to what extent the project was
responsible for these outcomes. At the micro-level, the project directly assisted the
two financial institutions to improve their policies, practices and systems that
directly benefited the poor, women and small producers. Although small in
resources compared to total assets of the financial institutions, the project
influenced development of financial products suitable for the beneficiaries. The two
financial institutions have shown a higher gender and poverty sensitivity in
selecting clients and giving priorities to households that could not offer any
collateral. All institutions appear to have strengthened their institutional capabilities
and services. Either for the whole provincial network or for the programme county
sub-branches, an improved capacity per outlet and service coverage were reported
which in turn led to an increase in the numbers of customers in the RCC’s savings
and credit business.
62. Based on the above assessment, the PPA rates project impact on institutions and
policies as moderately satisfactory (4).
Overall impact on rural poverty reduction
63. Overall impact. Despite some issues related to attribution, it is clear that the
project showed positive impacts. Based on the above analysis, the overall rural
poverty impact is rated satisfactory (5).
19
C. Other performance criteria
Sustainability 64. Definition. The sustainability of rural financial service program may simply be
defined as the continuation of services without depletion of the credit funds and
with generation of profits. From demand-side, it is important to see clients make
profits from businesses financed by credit from respective financial institution and
that there is continued demand for financial services. From the supply side, it is
important to ensure that the financial service provider (financial institution) is a)
institutionally and financially sustainable. Several indicators such as i) governance,
ii) policies, iii) capacity to offer demand-driven services, iv) strong internal system
and control and v) capacity of human resources, are used to assess institutional
sustainability. On the other hand, financial viability is measured by i) profitability,
ii) level of non-performing loan, iii) return on equity, and iii) return on assets. The
following sections present sustainability of the RFSP, or, rather, the sustainability
of the two implementing financial institutions, using the above definition.
65. Demand side (sustainability of clients). Proxy indicators are used to assess the
sustainability of clients. The RIMS surveys should have analysed profitability of
sample business/income-generating activities financed by the project. In the
absence of such direct measure, the following observations can be made about the
financial sustainability of clients:
RIMS surveys recorded that households in the project villages have
increased household’s assets as well as reduced poverty level, that is, lesser
number of households are now below poverty. Notwithstanding
contributions from exogenous factors, access to financial services increased
their income, which is a reflection of profits from farming and other income
generating activities.
Clients have repaid loans on time as noted from interviewing staff as well as
sample clients, which is also a proxy indicator for profitability as well as
improved family cash-flow. In addition, the non-performing loans are
declining, which indicates good repayment of loans.
Clients are borrowing larger loans in successive years, which reflect their
capacity to manage larger loans as well as interest from borrowing. In
addition, the number of new clients is increasing, which indicates demand
for financial services.
Interviews with sample clients and staff reveal that business opportunities
are expanding in rural areas, more and more profitable farming
opportunities are emerging that will create demand for financial services.
Within the project period, the financial institutions provided loans to more
than 32,000 clients. Both are continuing with their services. CRCBank
reports over 7,000 new or returning clients in the period 2010-2012 (April),
a clear indication of demand.
66. Overall, it can be concluded that clients are making profits from business/farming
and demand for financial services is increasing over the years, which is expected to
increase in future as well.
67. Supply side - Institutional sustainability of the financial institutions. The
following aspects indicate strong institutional sustainability of PRCCU/CRCBank:
Clear legal ownership and governance;
Both institutions have elaborated savings and credit policies and developed
many credit products that have been rolled out in the project as well as in
other counties. Both financial institutions have developed the internal
capacity to design and roll out new products as demonstrated during the
project period;
20
Both institutions have strong internal management and control systems to
oversee overall performance; and
Both institutions benefited from training courses conducted under the
project. More than 5,000 officials were trained. More importantly, training
courses were designed and conducted by internal persons, which
demonstrate strong in-house capacity.
68. Financial viability of the financial institutions. The following indicators show
that both financial institutions are financially viable to offer sustainable financial
services:
Profitability: Both financial institutions for the last several years consistently
posted profits and expanded businesses. Profit margins for CRCBank and
RCCU are given in table 9A and table 9B;
As part of restructuring and improved performance, both financial
institutions have successfully reduced non-performing loans. In 2011, and
using the new and less strict risk categories, the non-performing loans for
CRCBank amounted to 1.44 per cent only; and
In spite of initial under-capitalization in particular in Shaanxi Province, both
institutions are now assessed by this PPA mission to be adequately
capitalized.
Table 9A Selected performance indicators in Shaanxi RCCU
Provincial RCCU Average programme county
Years 2005 2009 2010 2011 2005 2009
Return on equity 16.00% 108.81% n/a n/a 22.60% 41.84%
Return on assets 0.45% 1.45% n/a n/a 1.48% 1.98%
Profit margin 11.17% 32.68% n/a n/a 26.31% 39.64%
Asset utilization 4.01% 4.43% n/a n/a 5.56% 5.09%
NPL 16.96% 12.94% n/a n/a 8.39% 5.72%
Financial self sufficiency 112.78% 152.05% n/a n/a 148.98% 157.67%
Source: Project Completion Report (2011) Table 9B Selected performance indicators in Chongqing RCB
Chongqing RCB Yunyang Yuyang
Years 2005 2009 2010 2011 2005 2009 2005 2009
Return on equity* 17.06% 14.56% 13.65% 15.17% 19.54% -- 14.05% --
Return on assets 0.55% 0.72% 1.26% 1.35% 0.6% 1.56% 0.63% 1.44%
Profit margin 11.15% 13.62% 38.75% 39.8% 12.22% 33.17% 10.72% 34.76%
Asset utilization 5.05% 6.25% n.a n.a 4.98% 4.7% 5.87% 4.13%
NPL 12.01% 3.63% 2.38% 1.44% 17% 3.26% 15.72% 9.77%
Financial self sufficiency
108.14% 148% 1631% 1490% 101.97% 147.44% 117.27% 140.86%
Source: Project Completion Report (2011)
* County level return on equity was no longer recorded in the RCB after its conversion to a single legal person bank for the provincial network.
69. Overall conclusions. Both Shaanxi RCCU and CRCBank are financially viable
institutions and the target population may expect to receive financial services way
21
beyond the project period. Based on the above assessment, sustainability is
considered to be satisfactory (5).
70. Exit strategy. The project had a built-in exit strategy as it was partnering with
two strong financial institutions. After completion of all non-finance type project
activities, the lines of credit are used as revolving funds by the two institutions.
The IFAD loan is expected to be repaid as per agreement. Meanwhile, both
institutions are continuing to provide services to the existing clients and taking in
new clients.
Innovation and scaling-up 71. The main innovation came from the development of loan products by both
CRCBank and Shaanxi provincial RCCU (later Cooperative Bank). CRCBank reported
the development of 35 loan products, of which 17 are related to agricultural
development. Similarly, Shaanxi PRCCU have developed new loan products. A
number of loan products have been reported to be highly popular among clients,
such as the loan for migrant workers and the farm product loan. These products
target agriculture and rural micro enterprise more than poor productive rural
households, but have had an impact in the two project areas regardless.
72. In Shaanxi, a product called “Women Starting Their Business” was developed. This
has led the county RCCUs to increase their lending to the segment of women
entrepreneurs. Shaanxi RCCU developed a special loan product for returning
migrant workers which helped returning migrants with a complementary financing
to support their resettlement in agriculture related income-generating activities, as
well as for the start-up of non-farm business ventures.
73. In Chongqing, programme RCCUs/RCB sub-branches after MTR recommendations
explored efficient ways of extending the credit services to grassroots level through
a mechanism of broker lending. By creating a network of farmer credit agents at
village level who acted as credit brokers between lenders and borrowers,
RCCUs/RCBs were able to lend to farmers in remote villages at a low operating
cost. Meanwhile, an incentive-driven credit screening by the brokers helped
lowering the associated risks and provided additional means of credit monitoring
and collection.
74. It should be recalled that the two financial institutions have in fact replicated and
introduced all these new loan products beyond the project counties. Both reported
that these loan products are available from all rural branches, sub-branches and
outlets. This is a good example of scaling-up. It has also been reported that CBRC
supported and promoted the loan product for natural resource management
(leasing of forest land for developing forestry products) beyond Chongqing
province. Overall, this criterion is considered satisfactory (5).
Gender equality and women’s empowerment
75. There was no explicit target for any gender specific activities. A number of
activities directly benefited women. For example, of the 3,260 trainees of
CRCBank, 27 per cent were women who received training in several areas.
Similarly, in PRCCU 15 per cent of trainees were women. A sample interviewed
during the PPA mission expressed their satisfaction about the contents of the
courses and mentioned that they are applying the gained knowledge in day to day
operations.
76. Of the 36,224 total clients reached by the financial institutions, 28 per cent were
women who in many cases are first time borrowers from any formal financial
institutions. They are currently interacting with the financial institutions, although
male members of the families still play a large role in securing loans. Women
clients were found actively participating in crop production and raising livestock
although marketing still remains a man’s domain. The financial institutions reported
22
large increase in savings accounts by women. Overall, this criterion is rated as
moderately satisfactory (4).
D. Performance of partners
Performance of IFAD
77. IFAD took a strong stance with regard to financial and institutional sustainability as
an important success factor in rural finance. The completion report recognises the
role of IFAD in providing continuous support and training opportunities for PMO
staff. Moreover, IFAD showed flexibility in adjusting its assistance and financial
allocation to cope with the changing environment given the nationwide reform
process of the rural finance system contributes to the positive assessment.
Notably, the RFSP was IFAD’s first sector programme and therefore could not rely
on previous experience. Hence, the project experienced a series of issues, most
notably: (i) the implied programme objective of policy consultation was perceived
as ambitious; (ii) the complexity of the RCC structure was underestimated; and
(iii) implementation of certain activities pre-MTR was idle due to the national
context. At MTR, the Fund addressed the lack of appropriate institutional
arrangements to implement a policy reform project by reallocating resources to
more practical components such as line of credit and guided the development of
innovative and pro-poor financial products. Direct supervision has led to more
timely and effective interventions to rectify problems arising during project
implementation, yet, at the same time, the supervision process is still over
dependent on the MTR as the point at which adjustments in projects are to be
made. Based on the above assessment, the performance of IFAD is considered
overall to be moderately satisfactory (4).
Performance of the Government
78. Ministry of Finance. Overall, management and coordination was handled by the
Ministry of Finance, that is, the IFAD partner ministry on behalf of the Government.
The PCR reports that the office at the Ministry of Finance responsible for IFAD
showed promptness in coordinating communications between IFAD and the two
programme provinces. Despite its frequent change of personnel in charge, most of
the applications and files were processed without delay, once they were sent by the
programme Department of Finance or IFAD. The government established a PMO in
each province, who were the actual coordinators for programme implementation.
Both PMOs were successful in mobilizing and coordinating the counterpart funding
for the programme, which exceeded the original 30 per cent in both provinces. Yet,
implementation progress was at times slow given the absence of full-time PMO
staff, a fact that also led to uneven monitoring and evaluation data and delayed
submission of progress reports. Provincial PMOs successfully assisted county PMOs
by providing implementation support at field level and promoting the RFSP at
various international and domestic events. The final PCR missed a discussion of
interest rate levels specifically for the types of loans that are relevant for the IFAD
target population and has no adequate and recent information on the quality of the
IFAD financed portfolio managed by the two RCC networks in Shaanxi and
Chongqing. Given that most of IFAD funds went for microcredit financing, the
absence of performance information of the IFAD credit funds in the PCR cannot be
justified.
79. Departments of Finance. The Departments of Finance in Shaanxi and Chongqing
managed the Special Account and Withdrawal Applications programme prepared by
PMOs for submission to IFAD. IFAD disbursements were timely transferred to the
PMOs in support of continued implementation. The Departments of Finance
respectively in Chongqing and Shaanxi were challenged by the multitude of donors’
projects they are responsible for, and therefore, the time allocation for the RFSP
was somehow limited. Especially in Chongqing, delays in processing withdrawal
applications before submission to IFAD occurred regularly and PMOs requests were
not always followed-up by the Department of Finance. In Shaanxi, the RFSP was
23
the second IFAD project in the province and the Department of Finance was more
familiar with IFAD proceeds. This had eased the processing of withdrawal
applications, communications and coordination with IFAD.
80. Overall, the performance of the Government is considered to be moderately
satisfactory (4).
E. Overall project achievement
81. The PPA mission rates the project’s overall achievement as satisfactory (5). The
project contributed to strengthening institutional, financial, and operational
capacity of the two implementing financial institutions – Chongqing Rural
Commercial Bank and Shaanxi RCCU network – that led to increased access to
finance by the poor, women and small producers. It also contributed to enhancing
livelihoods of rural households in the project counties. In particular, impact studies
showed increases in household assets and production, improvement in food
security, and strengthening of human and social capital and socio-economic status
of women, although not all impacts on households can be fully attributed to the
project alone. The project partners developed new financial products and rolled
them beyond project counties and encouraged them to continue serving the small
clients. The two financial institutions are institutionally and financially viable to
continue offering financial services to poor smallholders. However, the project
originally designed as a sector project to participate in systemic policy reforms led
by CBRC could not do so due to a lack of proper institutional arrangements, rapid
changing nature of the on-going reform process and inadequate grasp of the
reform issues and institutional complexity of the rural finance sector in China. The
project concept was relevant but design was ambitious and weak in institutional
arrangements.
IV. Conclusions and recommendations
Conclusions
82. The following lessons may be drawn from the implementation experience of RFSP
for future IFAD project in China and elsewhere:
(a) Design process and institutional arrangement. The design team did not do
proper in-depth consultations with various stakeholders, especially at the
national/central level relevant institutions mandated to lead the rural finance
sector. Some discussions were held and contacts were made but those
discussions did not lead to formal agreement with CBRC to be the lead national
implementing agency of the project. Without having the central/national agency
as the host and implementer of the project, it is hard to imagine how the
project could have contributed to systemic change in the sector. There was no
formal feedback system to pass on the knowledge generated by the project to
the policy makers. CBRC independently made all policy decisions to implement
the nation-wide RCCU networks including the two RFSP partner financial
institutions. Three critical lessons are: a) for a policy reform project to drive
systemic changes it has to be hosted and managed by the central body
responsible for taking policy reforms; and b) all consultations and dialogue
during design process must be held at appropriate level and lead to formal
agreement about scope of works as well as responsibilities of the parties
involved so that binding commitment to the project is ensured; and c) involving
a banking supervisory agency as an implementing agency for a pro-poor
financial sector reform programme is counterproductive since this design
introduces conflict-of-interest issues between the same agency as regulator and
supervisor of financial institutions on one hand, and as implementer of reform
on the other.
(b) Understanding implementing institution’s work process. It is also critical
that the design process and design documents reflect the core business, scope
24
and mandate, strengths and weaknesses, and internal work process of the
implementing agencies. The two financial institutions performed well in
disbursement and recovery of loans, developing new products and activities
related to reaching the clients, which were within their mandate and expertise.
On the other hand, it was difficult for them to handle the procurements of
several advisory services. At the same time, none of the two local financial
institutions was in a position to influence central policies. Their job was to
implement in letter and spirit the central regulations and policies by CBRC. It
was an undue expectation on the part of the project to ask the two provincial
RCCs to experiment with new ideas and influence national policy. It is critical
that the design team/process understands the partner institutions adequately
and that their concerns are addressed at the design stage even if that may
mean a longer design process.
(c) IFAD financed portfolio performance needs continued and priority
attention. The absence of performance data on the IFAD financed loan
portfolio in Shaanxi and Chongqing needs to be noted. Comparative information
in the PCR date back to a single static figure for end FY 2007 without the
possibility to assess portfolio performance over time and in comparison between
the two project locations. This discussion should also have included the criteria
used to determine impaired portfolios and have provided a critical discussion on
the adequacy of the same.
(d) Partnership with viable financial institutions. One more lesson to be
learned from the RFSP is that, to ensure sustainable financial service delivery to
the poor and small producers, IFAD must select strong and financially viable
financial institutions as partners. The positive achievements by the project were
largely due to selection of two strong and resourceful institutions with financial
services as their core business.
Recommendations
83. Future direction of rural finance reform. There is a need and scope for future
reform in the rural finance sector in China. A number of issues, such as lack of
strong competition in the rural areas (RCCs monopolize rural areas), potential
effectiveness of village and township banks as they are initiated by RCCs or RCBs,
ownership structure of various institutions and prudential regulatory issues,
strengthening of governance are all areas where further research and reforms are
needed. However, it is unclear when, if at all, further reforms in these areas will be
embarked upon by CBRC. IFAD should maintain regular interaction with CBRC and
the Ministry of Finance to explore possibility of participating in future rural finance
sector reforms. Concrete proposals as to how to structure a policy type of loan
from IFAD’s perspective were included in the MTR and are still valid.
84. With fast economic growth the need for small loans is expected to decline in
prospering rural areas, but the demand of loans for farming is expected to
continue. Financial services in remote and isolated areas as well as in central and
western provinces are needed for poverty reduction. Therefore, expansion of
financial services as well as developing appropriate institutional arrangements for
these areas are important in an environment where RCCs and RCBs are
increasingly driven by profit motives and inevitably are moving away from less
profitable areas and businesses.
85. IFAD has three on-going projects with rural finance components in China, which
were designed after RFSP. All these projects are implemented through either RCCs
or women federations in central and western provinces, which are poorer,
compared to eastern provinces. IFAD’s assistance in Chongqing and Shaanxi will no
longer be needed as the rural economy in both provinces is growing fast and both
financial institutions are institutionally and financially capable of offering financial
services to the target population. In this context, future IFAD interventions should
25
continue developing and expanding financial services in poorer provinces and
remote communities. Lastly, the RIMS and other impact studies should include
profitability analysis of sample income generating activities financed by IFAD
projects. This would be useful to document increases in income from income-
generating activities and the contribution of such income to total family annual
income. Such analysis will provide learned answers while assessing the direct
impact of financial services to the poor.
Annex I
26
Rating comparison
Criteria IFAD-PMD ratinga PPA rating
a Rating disconnect
Project performance
Relevance 4 4
Effectiveness 4 5 +1
Efficiency 4 4
Project performanceb 4 4
Rural poverty impact
Household income and assets 5 5
Human and social capital and empowerment 4 4
Food security and agricultural productivity 5 5
Natural resources environment and climate change n.a. n.a
Institutions and policies 4 4
Rural poverty impactc 5 5
Other performance criteria
Sustainability 4 5 +1
Innovation and scaling up 4 5 +1
Gender equality and women’s empowerment 4 4
Overall project achievementd 4 5 +1
Performance of partnerse
IFAD 5 4
Government 4 4
Average net disconnect
a Rating scale: 1 = highly unsatisfactory; 2 = unsatisfactory; 3 = moderately unsatisfactory; 4 = moderately satisfactory; 5 =
satisfactory; 6 = highly satisfactory; n.p. = not provided; n.a. = not applicable. b Arithmetic average of ratings for relevance, effectiveness and efficiency. c This is not an average of ratings of individual impact domains.
d This is not an average of ratings of individual evaluation criteria but an overarching assessment of the project, drawing
upon the rating for relevance, effectiveness, efficiency, rural poverty impact, sustainability, innovation and scaling up, and gender. e The rating for partners’ performance is not a component of the overall assessment ratings.
Annex II
27
Basic project data
Approval (US$ m) Actual (US$ m)
Region Asia and the Pacific Total project costs 21.3 30.4
Country People’s Republic
of China IFAD loan and percentage of total 14.7 69% 15.9 52%
Loan number 634 Borrower 0.406 1.9%
Type of project (subsector) CREDI
Rural Credit Cooperatives (loan) 6.1 28.6%
Financing type E
Rural Credit Cooperatives Unions (grant) 0.097 0.46%
Lending termsa HC Cofinancier 3
Date of approval 21 April 2004 Cofinancier 4
Date of loan signature 27 May 2004
China Banking Regulatory Comission (grant) 0.024 0.12%
Date of effectiveness 13 September 2005 Other sources:
Loan amendments 1
Number of beneficiaries
(if appropriate, specify if direct or indirect)
120,000 rural households
Direct: 36,224 households
Indirect:
830,584 households
Loan closure extensions 1
Country programme managers
E. Martens
T. Rath
S. Jatta Loan closing date 31 March 2010 30 September
2010
Regional director(s) T. Elhaut Mid-term review April 2007
Project completion report reviewer L. Kellens
IFAD loan disbursement at project completion (%) 95.74%
Project completion report quality control panel
A. Brubaker
A. Lambert Date of project completion report February 2011
Source: Project Completion report a There are four types of lending terms: (i) special loans on highly concessional terms, free of interest but bearing a service
charge of three fourths of one per cent (0.75%) per annum and having a maturity period of 40 years, including a grace period of 10 years; (ii) loans on hardened terms, bearing a service charge of three fourths of one per cent (0.75%) per annum and having a maturity period of 20 years, including a grace period of 10 years; (iii) loans on intermediate terms, with a rate of interest per annum equivalent to 50% of the variable reference interest rate and a maturity period of 20 years, including a grace period of 5 years; (iv) loans on ordinary terms, with a rate of interest per annum equivalent to one hundred per cent (100%) of the variable reference interest rate, and a maturity period of 15-18 18 years, including a grace period of three years.
Annex III
28
Terms of reference
I. Background 1. The Independent Office of Evaluation of IFAD (IOE) will conduct a project
performance assessment (PPA) of the Rural Finance Sector Programme (RFSP) in
China The PPA is a project-level evaluation aiming at: (i) providing an independent
assessment of the results and impact of the project / programme under
consideration; and (ii) generating findings and recommendations for the design and
implementation of on-going and future operations in the country.
2. PPAs are conducted on a sample of projects for which a project completion report
(PCR) has been validated by IOE and taking into consideration the following
criteria: (i) synergies with forthcoming or on-going IOE evaluations; (ii) major
information gaps in the PCR; (iii) novel approaches; and (iv) geographic balance.
In the case of RFSP, a project completion report validation was undertaken in 2011
and forms the basis for this PPA exercise.
3. The PPA applies the evaluation criteria outlined in the IFAD Evaluation Manual. In
view of the time and resources available, the PPA is generally not expected to
undertake quantitative surveys; rather, it adds analysis based on interviews at
IFAD headquarters, interactions with stakeholders in the country including project
beneficiaries, and direct observations in the field.
4. Country context. Over the past 30 years, China has emerged as one of the
world’s leading powers, and it boasts the world’s second largest economy. The
government has stepped up investments in rural areas to meet the growing market
demand for agricultural products and to improve the livelihoods of rural people.
One result has been a dramatic decrease over the last 30 years in the number of
people living in absolute poverty. China’s GDP was nearly US$5 trillion in 2009
(almost double compared to 2006), and annual GDP per capita growth reached
8.5% in 2009. This positive economic climate, in collaboration with the on-going
rural finance reform, challenges the attribution of the rural poverty impact
measured in the programme area.
5. Project description. The Rural Finance Sector Programme (RFSP) was designed
with the overall objective to ensure that rural financial services contribute
effectively and sustainably to reducing poverty. Its specific objectives were to
ensure that: (i) rural households, including poor households, have better access to
financial services and effectively make use of them to improve their living
standards; (ii) Rural Credit Cooperatives (RCC) policy reforms have been
successfully tested in the programme area and are being implemented in IFAD-
financed interventions elsewhere; (iii) improved institutional and operational
management capacity in programme RCCs is applied on a larger scale and
contributes to improving cost-effectiveness and profitability; and (iv) modalities to
resolve the problem of non-performing loans have been tested and applied on a
wider scale.
6. The programme was to be implemented at the national level to support the China
Banking Regulatory Commission (CBRC) and the Rural Credit Cooperatives Unions
(RCCUs) in testing a number of policy and institutional reforms. Those tests were
to be undertaken in selected geographical areas before applying the reforms at a
national scale. Design provided for pilot activities in the project areas of on-going
IFAD-funded interventions in order to ensure that microfinance policies are
complemented by both agricultural and social support services and infrastructure
development. The programme would initially be implemented in Chongqing and
Shaanxi Provinces.
Annex III
29
7. The programme was expected to benefit about 120,000 poor rural households.
Several of the above-mentioned policy reforms were expected to improve the
access of poor households to loans, especially in terms of micro and group lending
and loans for women. Other policies were to help to transform the RCCs into
sustainable financing institutions, which was in the interest of all their clients but
especially the poor for whom RCCs constitute the only source of formal deposits
and loans. The programme would support those policy reforms and assist in on-
going or planned policy adjustments with a view to help refining implementation
modalities and taking care that new policies have adequate poverty and equal
access/gender dimensions.
8. The programme comprised five components:
(a) Policy development (11 per cent of total base costs) which focuses on as-
sisting on-going or planned policy adjustments, refining implementation mo-
dalities, and ensuring that new policies have clear gender and poverty dimen-
sions;
(b) Institutional development (19 per cent of base cost) focusing on infra-
structure and equipment for RCCs, staff training and the legal structure;
(c) Operational development (4 per cent of base cost) to support the account-
ing system and policies and the staff incentive system;
(d) Financing (60 per cent of base cost) through the provision of credit lines to
help catalyse innovative microfinance practices and to demonstrate the opera-
tional viability of well-designed credit programmes to the RCCs; and
(e) Programme management (6 per cent of base cost).
9. The principal programme partners were the networks of RCCs that were federated
up to the province level, but do not have a national apex. The CBRC and provincial
RCCUs in the two programme provinces of Chongqing and Shaanxi were the
principal implementation partners of the programme. Programme Management
Offices (PMOs) were established within provincial RCCUs and were expected to
receive strategic and policy guidance related to RFSP from the CBRC, and to
manage and coordinate operations undertaken by county RCCUs at field level. A
National coordination and monitoring office (NCMO), responsible for providing
policy guidance, was planned to be established and located at the CBRC head
office.
II. Methodology 10. Objectives. The main objectives of the PPA are to: (i) assess the results of the
programme; and (ii) generate findings and recommendations for the design and
implementation of on-going and future operations in China.
11. Scope. The PPA will take account of the preliminary findings of the PCRV and
further desk review, issues emerging from interviews at IFAD headquarters, and a
focused mission to the country for the purpose of generating a comprehensive,
evidence-based evaluation. However, the PPA will not need to examine or re-
examine the full spectrum of programme activities, achievements and drawbacks,
but will focus on selected key issues. Furthermore, subject to the availability of
time and budgetary resources, due attention will be paid to filling in the major
evaluative information gaps of the PCR and other programme documents.
12. Evaluation criteria. In line with the evaluation criteria outlined in IOE’s Evaluation
Manual (2009), added evaluation criteria (2010)6 and the IOE Guidelines for PCRV
and PPA (January 2012), the key evaluation criteria applied in this PPA will include:
i. Relevance, which is assessed both in terms of alignment of project objec-
tives with country and IFAD policies for agriculture and rural development
6 Gender, climate change, and scaling up
Annex III
30
and the needs of the rural poor, as well as project design features geared to
the achievement of project objectives;
ii. Effectiveness, which measures the extent to which the project’s immediate
objectives were achieved, or are expected to be achieved, taking into account
their relative importance;
iii. Efficiency, which indicates how economically resources/inputs are converted
into results;
iv. Rural poverty impact, which is defined as the changes that have occurred
or are expected to occur in the lives of the rural poor (whether positive or
negative, direct or indirect, intended or unintended) as a results of develop-
ment interventions. Five impact domains are employed to generate a compo-
site indication of rural poverty impact: household income and assets; human
and social capital and empowerment; food security and agricultural produc-
tivity; natural resources, environment and climate change; and institutions
and policies;
v. Sustainability, indicating the likely continuation of net benefits from a de-
velopment intervention beyond the phase of external funding support. It also
includes an assessment of the likelihood that actual and anticipated results
will be resilient to risks beyond the project’s life;
vi. Innovation and scaling up, assessing the extent to which IFAD develop-
ment interventions have introduced innovative approaches to rural poverty
reduction and the extent to which these interventions have been (or are likely
to be) replicated and scaled up by government, private sector and other
agencies;
vii. Gender equality and women’s empowerment, This criterion is related to
the relevance of design in terms of gender equality and women’s empower-
ment, the level of resources committed, and changes promoted by the pro-
ject; and
viii. Performance of partners, including the performance of IFAD and the Gov-
ernment, will be assessed on an individual basis, with a view to the partners’
expected role and responsibility in the project life cycle.
13. Data collection. The PPA will be built on the initial findings of the PCRV. For
further information, interviews will be conducted both at IFAD headquarters and in
China. In the course of the in-country mission, additional primary and secondary
data will be collected in order to reach an independent assessment of performance
and results. Data collection methods will mostly include qualitative participatory
techniques. The methods deployed will consist of individual and group interviews,
focus group discussions with beneficiaries, and direct observations. The PPA will
also make use – where applicable – of additional data available through the
programme’s monitoring and evaluation system. Triangulation will be applied to
verify findings emerging from different information sources.
14. Stakeholders’ participation. In compliance with the Evaluation Policy of 2011,
the main programme stakeholders will be involved throughout the PPA. This will
ensure that the key concerns of the stakeholders are taken into account, that the
evaluators fully understand the context in which the programme was implemented,
and that opportunities and constraints faced by the implementing institutions are
identified. Regular interaction and communication will be established with the Asia
and the Pacific Division (APR) of IFAD and with the Government of China. Formal
and informal opportunities will be explored during the process for the purpose of
discussing findings, lessons and recommendations.
Annex III
31
III. Evaluation process 15. In all, the PPA will involve five phases: desk work; in-country work; report drafting
and peer review; receipt of comments from APR and the Government of China; and
the final phase of communication and dissemination.
16. Desk work phase. The related PCRV for RFSP and further desk review based on
official project documentation and other evaluative material as appropriate will
provide initial findings and identify key issues to be investigated by the PPA.
17. Country work phase. The PPA mission is scheduled for 31 May – 12 June 2012,
subject to the agreement by the Government of China. Mission members will
interact with the Government, local authorities, local partners, NGOs, programme
staff and clients (beneficiaries), and collect information from the programme’s
monitoring and evaluation system and other sources. At the end of the mission, a
brief will be provided to the IFAD partner ministry, followed by a wrap-up meeting
in Beijing, to summarize the preliminary findings and discuss key strategic and
operational issues.
18. Report drafting and peer review. At the conclusion of the field visit, a draft PPA
report will be prepared and submitted to IOE internal peer review for quality
assurance. Konstantin Atanesyan, Senior Evaluation Officer, together with another
evaluation officer to be designated, will be the peer reviewers for the PPA.
19. Comments by APR and the Government. The PPA report will be shared with APR
and thereafter with the Government for comments. IOE will finalize the report
following receipt of the Government’s comments.
20. Communication and dissemination. The final report will be disseminated among key
stakeholders and the evaluation report published by IOE, both online and in print.
IV. Key issues for investigation
21. According to the PCRV’s findings, programme support to reforming the rural
finance networks in Chongqing and Shaanxi has contributed to strengthened
institutional, financial, operational and technical sustainability of those institutions.
Improved access to financial services has impacted on the livelihoods of rural
households in the programme counties, although to a less extent that would have
been expected from a programme of such scope. Rural household assets and
production improved, as well as food security, human and social capital was
strengthened and the socio-economic status of women improved. RFSP did bring
innovative approaches in segmenting the lending market and adapting lending
products for the poor and women. However, the programme main weakness can be
found in the visionary but ambitious design, the weak poverty focus at the lowest
level, and the absence of clear linkages with other IFAD-funded projects.
22. In reaffirming the good overall quality of the PCR in presenting and analysing the
programme results, IOE found that, while rightly assessing project performance
against the evaluation criteria adopted by PMD and IOE, the report concentrated on
the final results, giving very little information about the changes that took place
during implementation and problems that had to be addressed and resolved. Also,
the report did not mention the reason for the 16 months gap between project
approval and loan signing. The PPA will therefore collect additional data in these
respects.
23. A review of the PCRV has led to the identification of a number of important issues,
some relevant to present challenges faced by IFAD. As such, the following issues,
among others, will be further investigated by the PPA.
24. Household income and net assets. Despite the implicit results chain of the
programme (improved access to financial services allowing rural households to
engage in income generating activities to improve their income), no specific data
Annex III
32
was collected regarding household income and income generating activities. Using
the RIMS data, the PCR suggest that income increased in the programme villages;
for example, in Chongqing, poor or low-income decreased from 16% in 2007 to
less than 5% in 2009; in Shaanxi, surveys recorded 11% of poor and low-income
in 2007, which was reduced to about 8% in 2009. However, these improvements
seem to correspond with the overall poverty reduction in the country (see
paragraph 4). The PPA will attempt to clarify this issue through data collection and
other evaluative methods.
25. Institutions and policies. The programme did not conduct the official policy
consultation as to the central regulatory agency as suggested by the Appraisal
report and the Mid-Term Review. The latter proposed an alternative policy
consultative mechanism after the retreat of CBRC to ensure that any systematic
innovations or operational breakthroughs obtained by the programme would be
forwarded to policy makers of sound practices in the national RCC reform.
However, the completion report does not provide more information or analysis on
why this was not done. The PPA mission will enquire on this aspect and report
accordingly.
26. Innovation and scaling up. The completion report indicates that the innovations
introduced by RFSP were not captured by PMOs and IFAD for replication and
scaling up in future project. The project design did not include an explicit
mechanism for further innovation, replication and scaling up, and this has not
occurred. Moreover, given the changed role of the CBRC, there was no apparent
mechanism to transfer knowledge and experience gained in provincial and country
level reform from this project to subsequent programme activities related to RCCs.
The PPA will analyse this specific aspect and provide recommendations for future
interventions.
27. Gender equality and women’s empowerment. Programme design targeted
poor households in the pilot counties, and provided for a Women Window so that
women could contract individual or group loans in their own names. However, this
activity did not deliver the expected results pre-MTR, mainly because of the lack of
adjusted loan products and the insufficient coverage of remote poor villages. The
post-MTR period focused on product development and promoted the introduction of
village credit models (which would further increase access for women). Despite
sporadic reports on this, there seemed to be no application of this village credit
model. The PPA mission will review this aspect and report accordingly.
V. Evaluation team 28. Under the guidance of Konstantin Atanesyan, Senior Evaluation Officer, Mr Mark
Keating, Evaluation Officer, has been appointed as Lead Evaluator for this PPA and
will be responsible for delivering the final report. He will be assisted by a senior
consultant as rural finance expert who will lead the mission and prepare the draft
report.
Annex IV
33
Methodological note on project performance assessments
A. What is a project performance assessment?1
1. The project performance assessment (PPA) conducted by the Independent Office of
Evaluation of IFAD (IOE) entails one mission of 7-10 days2 and two mission
members.3 PPAs are conducted on a sample of projects for which project
completion reports have been validated by IOE, and take account of the following
criteria (not mutually exclusive): (i) synergies with forthcoming or on-going IOE
evaluations (e.g. country programme or corporate-level evaluations); (ii) major
information gaps in project completion reports (PCRs); (iii) novel approaches; and
(iv) geographic balance.
2. The objectives of the PPA are to: assess the results and impact of the project under
consideration; and (ii) generate findings and recommendations for the design and
implementation of on-going and future operations in the country involved. When
the PPA is to be used as an input for a country programme evaluation, this should
be reflected at the beginning of the report. The PPA is based on the project
completion report validation (PCRV) results, further desk review, interviews at IFAD
headquarters, and a dedicated mission to the country, to include meetings in the
capital city and field visits. The scope of the PPA is set out in the respective terms
of reference.
B. Preparing a PPA
3. Based on the results of the PCRV, IOE prepares brief terms of reference (ToR) for
the PPA in order to sharpen the focus of the exercise.4 As in the case of PCRVs,
PPAs do not attempt to respond to each and every question contained in the
Evaluation Manual. Instead, they concentrate on the most salient facets of the
criteria calling for PPA analysis, especially those not adequately explained in the
PCRV.
4. When preparing a PPA, the emphasis placed on each evaluation criterion will
depend both on the PCRV assessment and on findings that emerge during the PPA
process. When a criterion or issue is not identified as problematic or in need of
further investigation, and no additional information or evidence emerges during the
PPA process, the PPA report will re-elaborate the PCRV findings.
Scope of the PPA
1 Extract from the PCRV and PPA Guidelines.
2 PPAs are to be conducted within a budget ceiling of US$25,000.
3 Typically, a PPA mission would be conducted by an IOE staff member with the support of a consultant (international
or national). An additional (national) consultant may be recruited if required and feasible within the evaluation budget. 4 Rather than an approach paper, IOE prepares terms of reference for PPAs. These terms of reference ensure
coverage of information gaps, areas of focus identified through PCRVs and comments by the country programme manager, and will concentrate the PPA on those areas. The terms of reference will be included as an annex to the PPA.
PCRV
assessment
PPA process
PPA ToR: Emphasis on selected criteria
and issues are defined
PPA report considers all criteria but
emphasizes selected criteria and issues
Annex IV
34
C. Evaluation criteria
5. The PPA is well suited to provide an informed summary assessment of project
relevance. This includes assessing the relevance of project objectives and of
design. While, at the design stage, project logical frameworks are sometimes
succinct and sketchy, they do contain a number of (tacit) assumptions on
mechanisms and processes expected to generate the final results. At the post-
completion phase, and with the benefit of hindsight, it will be clearer to the
evaluators which of these assumptions have proved to be realistic, and which did
not hold up during implementation and why.
6. For example, the PPA of a project with a major agricultural marketing component
may consider whether the project framework incorporated key information on the
value chain. Did it investigate issues relating to input and output markets
(distance, information, monopolistic power)? Did it make realistic assumptions on
post-harvest conservation and losses? In such cases, staff responsible for the PPA
will not be expected to conduct extensive market analyses, but might consider the
different steps (e.g. production, processing, transportation, distribution, retail)
involved and conduct interviews with selected actors along the value chain.
7. An assessment of effectiveness, the extent to which a project’s overall objectives
have been achieved, should be preferably made at project completion, when the
components are expected to have been executed and all resources fully utilized.
The PPA considers the overall objectives5 set out in the final project design
document and as modified during implementation. At the same time, it should be
flexible enough to capture good performance or under-performance in areas that
were not defined as an objective in the initial design but emerged during the
course of implementation.
8. The PPA mission may interview farmers regarding an extension component, the
objective of which was to diffuse a certain agricultural practice (say, adoption of a
soil nutrient conservation technique). The purpose here would be to understand
whether the farmers found it useful, to what extent they applied it and their
perception of the results obtained. The PPA may look into reasons for the farmers’
interest in new techniques, and into adoption rates. For example, was the
extension message delivered through lectures? Did extension agents use audio-
visual tools? Did extension agents engage farmers in interactive and participatory
modules? These type of questions help illustrate why certain initiatives have been
conducive (or not conducive) to obtaining the desired results.
9. The Evaluation Manual suggests methods for assessing efficiency, such as
calculating the economic internal rate of return,6 estimating unit costs and
comparing them with standards (cost-effectiveness approach), or addressing
managerial aspects of efficiency (timely delivery of activities, respect of budget
provisions). The documentation used in preparing the PCRV should normally
provide sufficient evidence of delays and cost overruns and make it possible to
explain why they happened.
10. As far as rural poverty impact is concerned, the following domains are
contemplated in the Evaluation Manual: (a) household income and assets;
(b) human and social capital and empowerment; (c) food security and agricultural
5 Overall objectives will be considered as a reference for assessing effectiveness. However, these are not always
stated clearly or consistent throughout the documentation. The assessment may be made by component if objectives are defined by components; however the evaluation will try to establish a correspondence between the overall objectives and outputs. 6 Calculating an EIRR may be challenging for a PPA as it is time consuming and the required high quality data are often
not available. The PPA may help verify whether some of the crucial assumptions for EIRR calculation are consistent with field observations. The mission may also help shed light on the cost-effectiveness aspects of efficiency, for example whether, in an irrigation project, a simple upgrade of traditional seasonal flood water canalization systems might have been an option, rather than investing on a complex irrigation system, when access to markets is seriously constrained.
Annex IV
35
productivity; (d) natural resources, the environment and climate change;7 and
(e) institutions and policies. As shown in past evaluations, IFAD-funded projects
generally collect very little data on household or community-level impact
indicators. Even when impact data are available, both their quality and the
methodological rigour of impact assessments are still questionable. For example,
although data report significant increases in household assets, these may be due to
exogenous factors (e.g. falling prices of certain commodities; a general economic
upturn; households receiving remittances), and not to the project.
11. PPAs may help address the “attribution issue” (i.e. establishing to what extent
certain results are due to a development intervention rather than to exogenous
factors) by:
(i) following the logical chain of the project, identifying key hypotheses and
reassessing the plausibility chain; and
(ii) conducting interviews with non-beneficiaries sharing key characteristics (e.g.
socio-economic status, livelihood, farming system), which would give the
mission an idea of what would have happened without the project
(counterfactual).8
12. When sufficient resources are available, simple data collection exercises (mini-
surveys) may be conducted by a local consultant prior to the PPA mission.9 Another
non-mutually exclusive option is to spot-check typical data ranges or patterns
described in the PCR by means of case studies (e.g. do PCR claims regarding
increases in average food-secure months fall within the typical ranges recorded in
the field?). It is to be noted that, while data collected by a PPA mission may not be
representative in a statistical sense, such data often provide useful reference points
and insights. It is important to exercise care in selecting sites for interviews in
order to avoid blatant cases of non-beneficiaries profiting from the project.). Sites
for field visits are selected by IOE in consultation with the government concerned.
Government staff may also accompany the PPA mission on these visits.
13. The typical timing of the PPA (1-2 years after project closure) may be useful for
identifying factors that enhance or threaten the sustainability of benefits. By that
stage, the project management unit may have been disbanded and some of the
support activities (technical, financial, organizational) terminated, unless a second
phase is going forward or other funding has become available. Typical factors of
sustainability (political support, availability of budgetary resources for
maintenance, technical capacity, commitment, ownership by the beneficiaries,
environmental resilience) can be better understood at the ex post stage...
14. The PPA also concentrates on IFAD’s role with regard to the promotion of
innovations and scaling up. For example, it might be observed that some
innovations are easily scaled up at low cost (e.g. simple but improved cattle-
rearing practices that can be disseminated with limited funding). In other cases,
scaling up may involve risks: consider the case of a high-yield crop variety for
which market demand is static. Broad adoption of the variety may be beneficial in
terms of ensuring food security, but may also depress market prices and thereby
reduce sale revenues for many households unless there are other, complementary
activities for the processing of raw products.
15. The PPA addresses gender equality and women’s empowerment, a criterion
recently introduced into IFAD’s evaluation methodology. This relates to the
emphasis placed on gender issues: whether it has been followed up during
7 Climate change criterion will be addressed if and when pertinent in the context of the project, as most completed
projects evaluated did not integrate this issue into the project design. 8 See also the discussion of attribution issues in the section on PCRVs.
9 If the PPA is conducted in the context of a country programme evaluation, then the PPA can piggy-back on the CPE
and dedicate more resources to primary data collection.
Annex IV
36
implementation, including the monitoring of gender-related indicators; and the
results achieve.
16. Information from the PCRV may be often sufficient to assess the performance of
partners, namely, IFAD and the government. The PPA mission may provide further
insights, such as on IFAD’s responsiveness, if relevant, to implementation issues or
problems of coordination among the project implementation unit and local and
central governments. The PPA does not assess the performance of cooperating
institutions, which now has little or no learning value for IFAD.
17. Having completed the analysis, the PPA provides its own ratings in accordance with
the evaluation criteria and compares them with PMD’s ratings. PPA ratings are final
for evaluation reporting purposes. The PPA also rates the quality of the PCR
document.
18. The PPA formulates short conclusions: a storyline of the main findings. Thereafter,
a few key recommendations are presented with a view to following up projects, or
other interventions with a similar focus or components in different areas of the
country.10
10
Practices differ among multilateral development banks, including recommendations in PPAs. At the World Bank, there are no recommendations but “lessons learned” are presented in a typical PPA. On the other hand, PPAs prepared by Asian Development Bank include “issues and lessons” as well as “follow-up actions” although the latter tend to take the form of either generic technical guidelines for a future (hypothetical) intervention in the same sector or for an ongoing follow-up project (at Asian Development Bank, PPAs are undertaken at least three years after project closure).
Annex V
37
Definition of the evaluation criteria used by IOE
Criteria Definitiona
Project performance
Relevance The extent to which the objectives of a development intervention are consistent with beneficiaries’ requirements, country needs, institutional priorities and partner and donor policies. It also entails an assessment of project design in achieving its objectives.
Effectiveness The extent to which the development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance.
Efficiency A measure of how economically resources/inputs (funds, expertise, time, etc.) are converted into results.
Rural poverty impactb Impact is defined as the changes that have occurred or are expected to occur in
the lives of the rural poor (whether positive or negative, direct or indirect, intended or unintended) as a result of development interventions.
Household income and assets
Household income provides a means of assessing the flow of economic benefits accruing to an individual or group, whereas assets relate to a stock of accumulated items of economic value.
Human and social capital and empowerment
Human and social capital and empowerment include an assessment of the changes that have occurred in the empowerment of individuals, the quality of grassroots organizations and institutions, and the poor’s individual and collective capacity.
Food security and agricultural productivity
Changes in food security relate to availability, access to food and stability of access, whereas changes in agricultural productivity are measured in terms of yields.
Natural resources, the environment and climate change
The focus on natural resources and the environment involves assessing the extent to which a project contributes to changes in the protection, rehabilitation or depletion of natural resources and the environment as well as in mitigating the negative impact of climate change or promoting adaptation measures.
Institutions and policies The criterion relating to institutions and policies is designed to assess changes in the quality and performance of institutions, policies and the regulatory framework that influence the lives of the poor.
Other performance criteria
Sustainability
The likely continuation of net benefits from a development intervention beyond the phase of external funding support. It also includes an assessment of the likelihood that actual and anticipated results will be resilient to risks beyond the project’s life.
Innovation and scaling up The extent to which IFAD development interventions have: (i) introduced innovative approaches to rural poverty reduction; and (ii) the extent to which these interventions have been (or are likely to be) replicated and scaled up by government authorities, donor organizations, the private sector and others agencies.
Gender equality and women’s empowerment
The criterion assesses the efforts made to promote gender equality and women’s empowerment in the design, implementation, supervision and implementation support, and evaluation of IFAD-assisted projects.
Overall project achievement This provides an overarching assessment of the project, drawing upon the analysis made under the various evaluation criteria cited above.
Performance of partners
IFAD
Government
This criterion assesses the contribution of partners to project design, execution, monitoring and reporting, supervision and implementation support, and evaluation. It also assesses the performance of individual partners against their expected role and responsibilities in the project life cycle.
a These definitions have been taken from the OECD/DAC Glossary of Key Terms in Evaluation and Results-Based Management
and from the IFAD Evaluation Manual (2009). b
The IFAD Evaluation Manual also deals with the “lack of intervention”, that is, no specific intervention may have been foreseen or
intended with respect to one or more of the five impact domains. In spite of this, if positive or negative changes are detected and can be attributed in whole or in part to the project, a rating should be assigned to the particular impact domain. On the other hand, if no changes are detected and no intervention was foreseen or intended, then no rating (or the mention “not applicable”) is assigned.
Annex VI
38
List of key persons met
Beijing
Mr Sun Yinhong, IFAD China Presence Officer
Ms Weijing Wang, IFAD Programme Officer
Mr Peter Situ, Rural Finance Specialist and author of PCR
Mr Zheng Kangbing, Head, Private Sector Coordination Unit, ADB, PRC
Mr Wang Jun, Lead Financial Specialist, The World Bank, China
Ms Wang Ying, Financial Sector Specialist, The World Bank, China Representative, Robbo
Bank
Dr Guangwen He, Professor and Director, Center for Rural Finance and Investment
Research, China Agricultural University
Ministry of Finance
Mr Zhang Lee, International Financial Institutional Div III, International Department
Re Chongqing Province
Mr Liu Zunming, Principal Staff, Chongqing Municipal Finance Bureau, External Finance
Division
Ms Jing Shu, General Manager, Head Office Company Business Office, Chongqing Rural
Commercial Bank
Ms Chen Yan, Vice General Manager, Corporate Banking Dept., Chongqing Rural
Commercial Bank
Ms Xin Gao, Board Office, Investor Relations Management
Ms Chen Xi, PMO
Branch Manager, CRCB, Younyang County
Credit Officers, CRCB, Younyang County
Project beneficiaries, Younyang County
Shaanxi Province
Mr Peng Hui, PMO, Shaanxi, PRCCU
Director, Shaanxi, PRCCU
Mr Wang, Chief of Xixian RCC
Chairman, Xixian RCC
Credit Officers, Xixian RCC
Beneficiaries, Xixian RCC
Annex VII
39
Bibliography
International Fund for Agricultural Development (IFAD) (1999). Country Strategic
Opportunities Paper.
____ (2001). Rural Financial Services in China, Thematic Study.
____ (2003). Appraisal Report.
____ (2004). Report and Recommendation of the President. Rural Finance Sector
Programme.
____ (2007). Mid Term Review.
____ (2008). Aide-Mémoire. Supervision mission May 2008.
____ (2009). Aide-Mémoire. Supervision mission April 2009.
____ (2009). Rural Finance Policy.
____ (2011). Project Completion Report. Main Report.
____ (2011). Project Completion Report. Appendixes 1 and 2.
____ (2011). Project Completion Report.
____ (2011). Country Programme Review.
____ (various years). Project Status Reports and Country Programme Issues Sheets.
IFAD/Independent Office of Evaluation (2010). Interim Evaluation Qinling Mountain Area
Poverty-Alleviation Project, People’s Republic of China.
____ (2010). Completion Evaluation West Guangxi Poverty-Alleviation Project, People’s
Republic of China.
____ (2011). Project Completion Report Validation.
United Nations Office for Project Services (UNOPS) (2006). Supervision Report.
____ (2007). Supervision Report.
P R O J E C T P E R F O R M A N C E A S S E S S M E N T
December 2013
IFAD Internal Printing Services
People's Republic of China
Rural Finance Sector Programme
International Fund for Agricultural Development
Via Paolo di Dono, 44 - 00142 Rome, ItalyTel: +39 06 54591 - Fax: +39 06 5043463E-mail: [email protected]/evaluationwww.ruralpovertyportal.org
ifad-un.blogspot.com
www.facebook.com/ifad
www.twitter.com/ifadnews
www.youtube.com/user/ifadTV
Independent Offi ce of Evaluation