document of the world bankdocuments.worldbank.org/curated/en/658281468236084058/...document of the...
TRANSCRIPT
Document of
The World Bank
Report No: ICR00001840
IMPLEMENTATION COMPLETION AND RESULTS REPORT
ON A
LOAN 7317-CL
IN THE AMOUNT OF US$25.13 MILLION
TO THE
REPUBLIC OF CHILE
FOR A
TERTIARY EDUCATION FINANCE FOR RESULTS PROJECT
IN SUPPORT OF THE FIRST PHASE OF THE TERTIARY EDUCATION FINANCE
FOR RESULTS PROGRAM
June 24, 2011
Human Development Sector Manager Unit
Bolivia, Chile, Ecuador, Peru and Venezuela Country Management Unit
Latin America and the Caribbean Regional Office
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
CURRENCY EQUIVALENTS
Currency Unit = Chilean Pesos
472= US$1
FISCAL YEAR
January 1 – December 31
ABBREVIATIONS AND ACRONYMS
AFD Direct Public Support (Aporte Fiscal Directo)
AFI Indirect public support
CAE State Guaranteed Student Loan Program (Crédito con aval del Estado)
CASEN Survey of National Socioeconomic Characterization (Encuesta de
Caracterización Socioeconómica Nacional)
CAN National Accreditation Commission
CFT Technical Training Centers
CPS Country Partnership Strategy
CRUCH Consejo de Rectores de Universidades Chilenas
DIVESUP Higher Education Division in the Ministry of Education (División de
Educación Superior)
FIAC Academic Innovation Fund
GoC Government of Chile
IBRD International Bank for Reconstruction and Development
ICR Implementation Completion Report
IPDP Indigenous People Development Plan
ICU Institutional Coordination Units
ISR Implementation Status Report
KPI Key Project Indicators
MECESUP Higher Education Improvement Project
Mece1 Higher Education Improvement Project (1998 - 2005)
Mece2 Higher Education Improvement Project (2006 - 2010)
Mece3 Higher Education Improvement Project (2011 - 2014)
M&E Monitoring & Evaluation
OECD Organization for Economic Co-operation and Development.
PAD Project Appraisal Document
PDO Project Development Objectives
PIU Project Implementation Unit
PSU University Admission Exams
SCT-Chile Academic Credit Transfer System of Chile
SIES Tertiary Education Information System
TEI Tertiary Education Institution
UCH University of Chile (Universidad de Chile)
UFRO Universidad de la Frontera
Vice President: Pamela Cox
Acting Country Director: Laura Frigenti
Sector Manager: Chingboon Lee
Project Team Leader: Michael Crawford
ICR Author: Roberta Malee Bassett
CHILE
TERTIARY EDUCATION FINANCE FOR RESULTS PROJECT
(First Phase Adaptable Program Lending)
CONTENTS
Data Sheet
A. Basic Information
B. Key Dates
C. Ratings Summary
D. Sector and Theme Codes
E. Bank Staff
F. Results Framework Analysis
G. Ratings of Project Performance in ISRs
H. Restructuring
I. Disbursement Graph
1. Project Context, Development Objectives and Design ............................................... 1
1.1 Context at Appraisal ............................................................................................. 1
1.2 Original Project Development Objectives (PDO) and Key Project Indicators
(KPI). .......................................................................................................................... 2
1.3 Revised PDO ......................................................................................................... 3
1.4 Main Beneficiaries ................................................................................................ 3
1.5 Original Components ............................................................................................ 4
1.6 Revised components ............................................................................................. 5
1.7 Other significant changes:..................................................................................... 5
2. Key Factors Affecting Implementation and Outcomes .............................................. 5 2.1 Project Preparation, Design and Quality at Entry ................................................. 5
2.2 Implementation ..................................................................................................... 6
2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization ...... 8
2.4 Safeguard and Fiduciary Compliance ................................................................... 9
2.5 Post-completion Operation/Next Phase .............................................................. 10
3. Assessment of Outcomes .......................................................................................... 11
3.1 Relevance of Objectives, Design and Implementation ....................................... 11
3.2 Achievement of Project Development Objectives (PDO) .................................. 11
3.3 Efficiency ............................................................................................................ 14
3.4 Justification of Overall Outcome Rating ............................................................ 15
3.5 Overarching Themes, Other Outcomes and Impacts .......................................... 15
3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops ... 17
4. Assessment of Risk to Development Outcome ......................................................... 17 5. Assessment of Bank and Borrower Performance ..................................................... 18
5.1 Bank Performance ............................................................................................... 18
5.2 Borrower Performance ........................................................................................ 19
6. Lessons Learned ....................................................................................................... 20
7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 21 Annex 1. Project Costs and Financing .......................................................................... 22 Annex 2. Outputs by Component ................................................................................. 23 Annex 3. Economic and Financial Analysis ................................................................. 30 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 36
Annex 5. Beneficiary Survey Results ........................................................................... 38
Annex 6. Stakeholder Workshop Report and Results ................................................... 39 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 40
Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 51 Annex 9. List of Supporting Documents ...................................................................... 52 MAP
i
A. Basic Information
Country: Chile Project Name:
Tertiary Education
Finance for Results
Project- First Phase
APL
Project ID: P088498 L/C/TF Number(s): IBRD-73170
ICR Date: 06/24/2011 ICR Type: Core ICR
Lending Instrument: APL Borrower: GOVERNMENT
Original Total
Commitment: USD 25.1M Disbursed Amount: USD 25.1M
Revised Amount: USD 25.1M
Environmental Category: C
Implementing Agencies:
Ministry of Education
Cofinanciers and Other External Partners:
B. Key Dates
Process Date Process Original Date Revised / Actual
Date(s)
Concept Review: 09/17/2004 Effectiveness: 11/15/2005 11/15/2005
Appraisal: 02/24/2005 Restructuring(s):
Approval: 07/05/2005 Mid-term Review: 01/19/2008 01/19/2009
Closing: 06/30/2009 12/31/2010
C. Ratings Summary
C.1 Performance Rating by ICR
Outcomes: Satisfactory
Risk to Development Outcome: Low or Negligible
Bank Performance: Moderately Satisfactory
Borrower Performance: Highly Satisfactory
C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)
Bank Ratings Borrower Ratings
Quality at Entry: Moderately
Satisfactory Government: Highly Satisfactory
Quality of Supervision: Moderately
Satisfactory
Implementing
Agency/Agencies: Highly Satisfactory
Overall Bank
Performance:
Moderately
Satisfactory Overall Borrower
Performance: Highly Satisfactory
ii
C.3 Quality at Entry and Implementation Performance Indicators
Implementation
Performance Indicators
QAG Assessments
(if any) Rating
Potential Problem Project
at any time (Yes/No): No
Quality at Entry
(QEA): None
Problem Project at any
time (Yes/No): No
Quality of
Supervision (QSA): None
DO rating before
Closing/Inactive status: Satisfactory
D. Sector and Theme Codes
Original Actual
Sector Code (as % of total Bank financing)
Central government administration 10 10
Tertiary education 90 90
Theme Code (as % of total Bank financing)
Administrative and civil service reform 20 20
Education for the knowledge economy 40 40
Managing for development results 40 40
E. Bank Staff
Positions At ICR At Approval
Vice President: Pamela Cox Pamela Cox
Country Director: Acting CD: Laura Frigenti Evangeline Javier
Sector Manager: Chingboon Lee Eduardo Velez Bustillo
Project Team Leader: Michael F. Crawford Lauritz B. Holm-Nielsen /
Kristian Thorn
ICR Team Leader: Michael F. Crawford
ICR Primary Author: Roberta Malee Bassett
F. Results Framework Analysis
Project Development Objectives (from Project Appraisal Document) The project development objective is to increase the effectiveness of public funding for
tertiary education by enhancing coherence, responsiveness, equity and quality in the
system, through stronger accountability for performance.
Revised Project Development Objectives (as approved by original approving authority)
There were no revisions made to the PDO during the implementation of the project.
iii
(a) PDO Indicator(s)
Indicator Baseline Value
Original Target
Values (from
approval
documents)
Formally
Revised
Target
Values
Actual Value
Achieved at
Completion or
Target Years
Indicator 1 : Number of students graduating from CFTs (Technical Training Centers) and IPs
(Professional Institutes) transferring to universities.
Value
quantitative or
Qualitative)
0 1000 2432
Date achieved 01/01/2004 12/31/2008 12/31/2010
Comments
(incl. %
achievement)
Target exceeded. Baseline revised in 2005 to 72 students transferring to CRUCH
universities. Final value corresponds to data available for all universities.
Indicator 2 : Socioeconomic profile of university first year students-- Q1/2.
Value
quantitative or
Qualitative)
Q1/2 16.1% Q1/2 19% Q1/2 24.7%
Date achieved 01/01/2000 12/31/2007 12/31/2009
Comments
(incl. %
achievement)
Target exceeded. Baseline revised in 2003 to 23.8%. Target not revised.
Indicator 3 : Socioeconomic profile of university first year students--Q3.
Value
quantitative or
Qualitative)
16.3% 19% 15.9%
Date achieved 01/01/2000 12/31/2007 12/31/2009
Comments
(incl. %
achievement)
Baseline revised in 2003 to19.8%. Target not revised.
Indicator 4 : Socioeconomic profile of university first year students-- Q4/5.
Value
quantitative or
Qualitative)
67.6% 62% 59.3%
Date achieved 01/01/2000 12/31/2007 12/31/2009
Comments
(incl. %
achievement)
Baseline revised in 2003to 56.4%. Target not revised.
Indicator 5 : Socioeconomic profile of CFT (Technical Training Centers) first year students--
Q1/2.
Value
quantitative or
Qualitative)
28.6% 33% 25.4%
Date achieved 01/01/2000 12/31/2007 12/31/2009
Comments
(incl. %
achievement)
Baseline revised in 2003to 42.5%. Target not revised.
iv
Indicator 6 : Socioeconomic profile of CFT (Technical Training Centers) first year students--
Q3.
Value
quantitative or
Qualitative)
27% 30% 28.8%
Date achieved 01/01/2000 12/31/2007 12/31/2009
Comments
(incl. %
achievement)
Baseline revised in 2003 to 26.5%.Target not revised.
Indicator 7 : Socioeconomic profile of CFT (Technical Training Centers) first year students--
Q4/5.
Value
quantitative or
Qualitative)
44.4% 37% 45.8%
Date achieved 01/01/2000 12/31/2007 12/31/2009
Comments
(incl. %
achievement)
Target exceeded. Baseline revised in 2003 to 31%. Target not revised.
Indicator 8 : First year drop-out rate in undergraduate education.
Value
quantitative or
Qualitative)
Available March 2006. 22.2%
Date achieved 03/01/2006 12/31/2009
Comments
(incl. %
achievement)
Baseline and targets not defined in PAD. In 2005, baseline of 19% established
for Cruch universities. Values achieved are: Cruch: 22.2%; Private Universities:
28.5%; Professional Institutes: 38%, and CFTs: 36%.
Indicator 9 : Percentage of indicators in performance agreements with a scoring at or above
the agreed target by institution.
Value
quantitative or
Qualitative)
0% 90% 52.5%
Date achieved 01/01/2005 12/31/2008 12/31/2010
Comments
(incl. %
achievement)
Indicator counts targets achieved or exceeded. When progress against targets is
measured, the average value achieved is 84% (University of Bío Bío: 98%,
University of La Frontera: 92%, University of Tarapaca: 80%, and University of
Chile: 67%).
(b) Intermediate Outcome Indicator(s)
Indicator Baseline Value
Original Target
Values (from
approval
documents)
Formally
Revised
Target Values
Actual Value
Achieved at
Completion or
Target Years
Indicator 1 : Perception of DIVESUP policy instruments by university leaders.
Value
(quantitative
or Qualitative)
Survey in 2005 End of project
survey.
Survey to be
conducted.
Date achieved 12/31/2005 12/31/2008 12/31/2010
v
Comments
(incl. %
achievement)
Survey to establish baseline was conducted in 2008. Follow-up survey expected
in 2011.
Indicator 2 : Number of studies on Tertiary Education conducted by date.
Value
(quantitative
or Qualitative)
0 4 7
Date achieved 05/19/2005 12/31/2008 12/31/2010
Comments
(incl. %
achievement)
Target exceeded. 7 studies were completed. Refer to Annex 2 for a list of these.
Indicator 3 : Percentage of undergraduate tertiary enrollees studying in accredited programs
and programs in the process of being accredited.
Value
(quantitative
or Qualitative)
31% 37% 51%
Date achieved 12/31/2003 12/31/2008 12/31/2010
Comments
(incl. %
achievement)
Target exceeded.
Indicator 4 : Percentage of masters degree tertiary enrollees studying in accredited programs
and in process of being accredited.
Value
(quantitative
or Qualitative)
12% 20% 35%
Date achieved 12/31/2002 12/31/2008 12/31/2010
Comments
(incl. %
achievement)
Target exceeded.
Indicator 5 : Percentage of doctoral degree tertiary enrollees studying in accredited programs
and in process of being accredited.
Value
(quantitative
or Qualitative)
22% 60% 85%
Date achieved 12/31/2002 12/31/2008 12/31/2010
Comments
(incl. %
achievement)
Target exceeded.
Indicator 6 : Number of sub-projects supported at the tertiary level (cumulative).
Value
(quantitative
or Qualitative)
30 120 371
Date achieved 01/01/2004 12/31/2008 12/31/2008
Comments
(incl. %
achievement)
Target exceeded. Target set at 40 subprojects per year in 2006, 2007 and 2008.
Actual was 94 subprojects in 2006, 158 in 2007 and 119 in 2008.
Indicator 7 : Number of sub- projects targeted at degree structure reform (cumulative).
Value 0 81 153
vi
(quantitative
or Qualitative)
Date achieved 01/01/2004 12/31/2008 12/31/2008
Comments
(incl. %
achievement)
Target exceeded. Target set at 27 subprojects per year in 2006, 2007 and 2008.
Actual was 25 subprojects in 2006, 64 in 2007 and 64 in 2008.
Indicator 8 : Enrollment in preparatory programs.
Value
(quantitative
or Qualitative)
0 3,900 35,713
Date achieved 01/01/2005 12/31/2008 12/31/2010
Comments
(incl. %
achievement)
Target exceeded. Baseline revised in 2005to 307 students. Of the value achieved,
75% of students were enrolled in Mecesup-supported preparatory programs.
Remainder were enrolled at independently established programs.
Indicator 9 : Relation between average PSU score for admission in teaching programs
supported and those not supported.
Value
(quantitative
or Qualitative)
No information available
for PAD.
5% increase from
the baseline. 4.1%
Date achieved 01/01/2005 12/31/2008 12/31/2009
Comments
(incl. %
achievement)
Target partially achieved. Baseline set in 2004 as 0.4%. Target achieved in 2006
(7.4%) and 2007 (5.7%), and not in 2008 (5.0%) and 2009 (4.1%).
Indicator 10 : Number of enrolled doctoral students in Chile.
Value
(quantitative
or Qualitative)
1,930 2,500 4,055
Date achieved 07/01/2003 12/31/2008 12/31/2010
Comments
(incl. %
achievement)
Target exceeded. Baseline revised in 2005 to 2,334. These value achieved
includes doctoral students from private universities, of which there were 501 in
2010.
Indicator 11 : Number of university academic staff enrolled in a PhD program in Chile or
overseas by field.
Value
(quantitative
or Qualitative)
2,861 3,100 893
Date achieved 01/01/2004 01/01/2008 01/01/2010
Comments
(incl. %
achievement)
Baseline revised to 860. Universities are hiring new staff with PhDs rather than
sponsoring the PhDs of existing staff.
Indicator 12 : Number of universities covered by a performance agreement.
Value
(quantitative
or Qualitative)
0 4 4
Date achieved 01/01/2005 12/31/2008 12/31/2010
Comments
(incl. %
achievement)
Target achieved. Non-MECESUP funded performance agreements have spread
to humanities programs at Universidad de la Frontera and Universidad de Chile.
Additional agreements to be negotiated in 2011 with the universities of
vii
Valparaíso, Talca, and Magallanes.
Indicator 13 : Percent of base funding (AFD) for institutions with a performance agreement
allocated through the agreement.
Value
(quantitative
or Qualitative)
0 13% 2.4%
Date achieved 05/19/2005 12/31/2008 12/31/2009
Comments
(incl. %
achievement)
Indicator calculated as MECESUP funds (numerator) over the sum of
MECESUP and Direct Fiscal Grant (AFD) funds. The AFD has grown more than
expected because it is tied to enrollment, which has grown exponentially.
G. Ratings of Project Performance in ISRs
No. Date ISR
Archived DO IP
Actual
Disbursements
(USD millions)
1 12/20/2005 Satisfactory Satisfactory 0.06
2 01/27/2006 Satisfactory Satisfactory 0.06
3 07/24/2006 Satisfactory Satisfactory 0.06
4 02/01/2007 Satisfactory Satisfactory 4.06
5 12/18/2007 Satisfactory Satisfactory 4.06
6 04/18/2008 Satisfactory Satisfactory 11.56
7 11/24/2008 Satisfactory Satisfactory 14.87
8 05/16/2009 Satisfactory Satisfactory 14.87
9 11/20/2009 Satisfactory Satisfactory 21.45
10 06/21/2010 Satisfactory Satisfactory 25.13
11 01/11/2011 Satisfactory Satisfactory 25.13
H. Restructuring (if any)
The Project was not restructured during implementation.
viii
I. Disbursement Profile
1
1. Project Context, Development Objectives and Design
1.1 Context at Appraisal
Country Background. Chile enjoys a reputation for high economic growth rates, but its ability to
grow and remain competitive has been uneven. Since 1998, growth rates have been only half of
the seven point one percent (7.1%) achieved from 1984-97. Total Factor Productivity, arguably
the most important measure of the quality and sustainability of growth, has dropped from an
annual 2.8% in that period to only 0.4% annually from 1998-2005. Returning to high levels of
economic growth is a critical policy concern for Chile.1 To do so, Chile needs to diversify and
raise productivity. Because the latter requires building the skills and knowledge necessary to
succeed in a highly competitive global environment, improving the quality of and expanding
access to tertiary education is at the core of the Government‘s strategy.
Sector Background: the Chilean tertiary education system. After two decades of
underinvestment accompanied by a rise in unregulated private provision, a consensus emerged to
invest in the rehabilitation of Chile‘s tertiary education institutions. The Government of Chile
(GoC) and the World Bank partnered to create the Program to Increase Quality and Equity in
Tertiary Education (Programa para el Mejoramiento de la Calidad y Equidad de la Educación
Superior), or MECESUP, as it was known. The 1998-2005 MECESUP project (hereafter Mece1)
invested in infrastructure and academic renovation in universities using a competitive fund with
merit-based allocation. It piloted a quality assurance system through the creation of accreditation
councils and sought to raise the critical mass of human capital available for Chile‘s expanding
tertiary enrollment by investing in domestic and foreign PhD training for university staff.
Ultimately, Mece1 became synonymous with serious higher education reform in Chile, and the
Project‘s ICR rated the achievement of its development objective as highly satisfactory.
After 2005, and building on the successes of Mece1, the principal policy goals for tertiary
education expanded to include introducing performance-based funding and continued expansion
of access and improvement of quality. It was hoped that these mechanisms could continue
progress against the system‘s principal challenges, which included:
Lack of coherence and articulation (i.e., no credit transfer system) between tertiary
education institutions (TEIs) and across tertiary education levels.
Considerable variation of quality of tertiary education and poor responsiveness to student
and labor market needs.
Inequities stemming from low participation rates and high dropout rates for students from
lower socioeconomic quintiles.
Low availability of advanced human capital, especially of PhDs to staff the expanding
system.
Insufficient accountability in the use of public funding and continued reliance on historical
criteria rather than on merit for funding allocation.
Lack of reliable data on the tertiary education sector.
1 Consejo Nacional de la Innovación para la Competitividad (CNIC), 2010.
2
Country Partnership Strategy (CPS) and rational for Bank Assistance. Mece1 showed the
value of the Bank as a catalyst and broker for key sector reform. In particular, the establishment
of a competitive funding mechanism for quality improvement at the program level based on
transparent selection criteria was an important innovation in Chilean tertiary education. Mece2
was a continuation of previous Bank operations in Chile and capitalized on their many lessons.
Having successfully supported ‗management for results‘ in selected sub-projects, the Bank was
in a key position to assist in the adoption of new performance-based financing mechanisms that
moved funding to the institutional level.
Higher level objectives to which the Project contributed
The Project formed part of the GoC‘s efforts to place Chile in a favorable position to realize the
benefits of the emerging knowledge economy. The goal was that the educational sector would
provide the skills and the knowledge necessary to sustain Chile‘s economic and social
development and enhance its national innovation capacity. To this end, the proposed Project, in
conjunction with the Science for the Knowledge Economy Project (P077282) and the Life-long
Learning and Training Project (P068271), formed a coherent set of policies and investments in
skills, knowledge and equity.
CPS-goals supported by the Project
The proposed Project was firmly rooted in the three pillars of intervention in the 2007-2011
Country Partnership Strategy with Chile. First, Mece2 added to the CPS goal of sustaining
overall economic growth and social progress in Chile. Second, it contributed to heightening
inclusion by improving tertiary learning opportunities for students from poor and/or indigenous
backgrounds. Finally, it supported the modernization of the state by (i) building capacity for
strategic policy formulation; (ii) strengthening sector monitoring and evaluation; and (iii)
increasing accountability for the use of public funding.
1.2 Original Project Development Objectives (PDO) and Key Project Indicators (KPI).
The Mece2 PDO is to increase the effectiveness of public funding for tertiary education by
enhancing coherence, responsiveness, equity and quality in the system, through stronger
accountability for performance.
Mece2 was conceived as a two-phase Adjustable Program Loan (APL). The Program
Development Objectives for the Phase 1 APL mirrored the Project Development Objectives,
although they were phrased in different terms. The Table below shows how Program and Project
objectives overlapped and how Key Project Indicators (KPI) corresponded to them. The PAD did
not include definitions of each objective, nor correspondence between objectives and indicators.
The below table was created by the ICR author.
3
Table 1. Program Objectives, Project Development Objectives, and Key Project Indicators
Program Objective for APL
Phase 1
PDO for APL Phase 1 Project:
Increase the effectiveness of
public funding by enhancing:
Key Project Indicators (KPI)
Increase GoC planning and
oversight in the tertiary
education sector
Coherence
Equity
Socio-economic profile of first
year students in tertiary
education
A. Improve policies and strategies
for tertiary education
Coherence
B. Consolidate a national system
for quality assurance
Quality
C. Improve national information
system for tertiary education
Accountability
Coherence
Responsiveness
D. Competitively fund academic
innovation
Coherence
Quality
Equity
Responsiveness
- First year dropout rate in
undergraduate education.
- Number of students from
CFT‘s and IPs transferring to
universities
E. Pilot performance-based
funding contracts
Accountability, Coherence,
Quality, Equity,
Responsiveness
Percent of indicators in
performance agreements met or
exceeded
The Project was to accomplish this through three main mechanisms:
New legislation, policy studies and initiatives for tertiary education.
A competitively-allocated small grants program (referred to as the Academic Innovation
Fund or FIAC for its Spanish acronym).
Four large results-based financing agreements (referred to as Performance Agreements).
1.3 Revised PDO
There were no revisions to the Project‘s Development Objectives during implementation.
1.4 Main Beneficiaries
The primary target groups benefitted mostly from the FIAC and the Performance Agreements.
Tertiary education institutions (TEIs) benefitted from increased investments in their degree
programs, better access to funds, more qualified staff, enhanced physical and online
infrastructure, more quality assurance, and greater ability to conduct research, carry out
special projects and perform monitoring and evaluation.
Academics and staff in TEIs benefitted from more investments in their human capital, an
increased ability to conduct research with better equipment, and more exchange
opportunities with academics in other universities both within and outside the country.
Current and future students of TEIs, particularly those that would not have attended—or
been able to stay—in tertiary education otherwise. This group benefitted from increased
quality of education, more remedial classes, shorter degree programs, more coherent
curricula, new and better PhD programs and teachers, more online course material, and
more exchange opportunities with other universities both within and outside the country.
4
The group also benefit from increased capacity in the Ministry of Education to manage
student financial aid.
Employers and society as a whole benefitted from the increased quality of tertiary education
graduates and production of knowledge throughout the country.
The secondary target group benefitted mostly from a strengthened institutional framework for
tertiary education.
Society as a whole, including think tanks, students, families, and Government, benefitted
from more and better quality information regarding tertiary education.
Ministry of Education benefitted from increases in the capacity provided by the Project
Implementation Unit (PIU), the investments to set up the quality assurance system, and the
strengthened student financial aid system.
1.5 Original Components
As noted in the PAD, the Project had two components: (i) strengthening institutional framework
for tertiary education and (ii) sub-projects to enhance coherence, efficiency, equity and quality.
Below each component are listed the subcomponents and their causal linkages to the PDO. These
linkages are the interpretation of the ICR author as they are not made explicitly in the PAD.
Project Component 1 (US$ 8.00 million)
Strengthened the capacity of the Ministry of Education and relevant agencies to develop policies,
strategies, and management skills to achieve the PDO.
Improvement of sector oversight and regulation. Partially targeted coherence by
strengthening the regulatory framework and the supervision mechanisms for the tertiary
education sector. Partially targeted equity by supporting the establishment of adequate
structures in the Ministry of Education for the administration of a new law on student
financial assistance. Loan disbursement for this subcomponent was conditioned on the
passing of the law.
Strengthening of policies and strategies for tertiary education. Targeted coherence by
financing studies of priority issues for post-secondary and university education.
Consolidation of national system for quality assurance. Targeted quality by supporting the
operation of the accreditation commissions for undergraduate and graduate tertiary
education. Loan disbursement for this subcomponent was conditioned on the passing of the
quality assurance law.
Establishment of a national observatory for tertiary education (later called the Sistema de
Información de Education Superior or SIES for its Spanish acronym). Targeted cohesion by
increasing the information available for decision-making. Targeted responsiveness by
increasing monitoring, evaluation and accountability. The quality assurance law made it
mandatory for TEIs to collect and report data to the Government.
Project management. Supported the operation of the team in charge of directing the
activities of component 1 and the two sub-components of component 2.
Project Component 2 (US$ 83.25 million)
Supported strategically targeted funds aimed at stimulating innovation and institutional
performance and increasing accountability for results. Funds were disbursed directly to
accredited TEIs who reported data to the SIES.
5
Innovative sub-projects for tertiary programs (FIAC). Targeted coherence, responsiveness,
equity and quality by stimulating innovation and strengthening tertiary programs.
Competitively allocated grants allowed TEIs to tactically address priorities in specific areas
of need. The latter centered on improving academic instruction and management quality,
specifically focusing on advancing human capital, academic innovation, improvement of
academic management, and curriculum reform.
Subprojects to stimulate institutional effectiveness in critical areas (Performance
Agreements). Targeted increased accountability and responsiveness for the use of public
subsidies in TEIs through four pilot performance-based financing agreements. Agreements
provided TEIs an incentive to implement plans of action in areas of high national priority, as
financing depended on their performance measured against agreed upon targets. Targeted
equity and quality by furthering, among others, remediation programs for poorly prepared
first-year students and higher quality pedagogy programs.
1.6 Revised components
The components were not revised during the course of the project.
1.7 Other significant changes:
The original closing date was extended from June 30, 2009, to December 31, 2010, in order to
finish implementing the subprojects from the FIAC and allow for discussions regarding a new
project (Mece3), particularly in light of the presidential elections in January 2010.
2. Key Factors Affecting Implementation and Outcomes
2.1 Project Preparation, Design and Quality at Entry
Soundness of the background analysis
The background analysis supporting the Project was sound and the rationale for the Bank‘s
intervention was unambiguous. The strategic relevance and approach of Mece2 was clear within
Chile‘s specific country context, and the technical aspects of the lending program evidenced
rigorous and thorough analysis. Lessons learned from the earlier Mece1 were incorporated into
Mece2‘s design. These included:
Identification of performance-based funding as the key means of overcoming the historical
inertia linked to stagnant quality.
Recognition that the small grant program (FIAC) under Mece1 did not sufficiently
strengthen core institutional management of TEIs nor build capacity within them.
Establishment of Performance Contracts sought to overcome this limitation.
Realization that funding physical infrastructure had declining returns after initial critical
investments were made. Greater returns would be had from focusing on ―soft‖ inputs like
curriculum quality and institutional management. Physical infrastructure was eliminated as
an eligible expenditure category in the Mece2 project.
Poor quality, partial information and data hampered the ability to make sound policy.
Without a legal mandate, institutions had no incentive to provide accurate data. This led to
the inclusion of a clause in the 2006 Quality Assurance Law that required provision of data
to the Ministry. Passing this legislation was a condition for Mece2 disbursement.
6
Assessment of the project design
The solidity of project design rests on the correct proportion of ―pilots‖ of performance contracts
(four) with the continuation and relative reduction of smaller competitive grants under the FIAC.
The mix demonstrated appropriate realism about the degree of complexity achievable and the
pace with which new reforms could be tested. The inclusion of a covenant for legislation to
consolidate the quality assurance system (by legally creating the new National Accreditation
Commission (Comisión Nacional de Acreditación—CNA) was also highly appropriate and
proved very successful. Project design also provided mechanisms to allow Mece2‘s staff to
facilitate high-level analytical work. This led, inter alia, to a series of high-profile studies by the
World Bank and the OECD on instrumental issues in tertiary education policy.
Project design could have improved in the articulation of the PDO with the selected indicators,
and the realism of the Project timeline.
The APL Program Objectives are phrased in such a way that they do not easily align with
the statement of the PDO. Moreover, the Key Performance Indicators are not clearly linked
to the specific Program/PDO objectives; a degree of inference is required.
The Project timeline did not sufficiently take into account the Borrower‘s regular cycle for
budgetary approval in Congress, where resources awarded one fiscal year can only be
disbursed the following year.
Adequacy of GoC‘s commitment, stakeholder involvement, and/or participatory processes
Government commitment during the preparation process was sound and continuous. Stakeholder
participation benefitted from the excellent reputation that the Mece1 project had and the deep
links it had forged with universities.
Assessment of risks and mitigation measures
Assessment of the risk to continued substandard information was accurate and more than
adequately remedied by the 2006 legislation requiring tertiary institutions to comply with
Ministry of Education data requests. Resistance to transparency, credit transfer systems, and
changes to the legal and regulatory framework were deemed mostly moderate and proved to be
so; none presented an impassable obstacle to implementation, although some resistance was
detected. Risks of organized resistance to the proposed changes, insufficient scale of the pilots,
or procurement transparency were also judged moderate and no significant problems
materialized around these risks.
2.2 Implementation
The Project was successfully implemented due to several enabling factors:
Strong commitment on behalf of the GoC facilitated the enabling environment for Mece2. In
2005 the GoC passed the Tertiary Education Student Financing Law and extensively financed
and promoted the expansion of the student aid system. This improved equity throughout the
country and provided TEIs with additional funds with which to upgrade their quality. In 2006 the
GoC shepherded the Quality Assurance Law through Congress and widely encouraged
institutional accreditation thereafter.
7
Competent and stable leadership and staff in the Project Implementation Unit. The PIU
conducted frequent formal and informal stakeholder consultations, and developed solid
relationships with stakeholders and partners. It maintained very fluid communications with TEIs,
quickly becoming a conduit between the Government and them. The PIU also made up for weak
institutional capacity in the Ministry of Education‘s Division of Higher Education, particularly in
the initial years of the project. In addition, the PIU:
Was open to suggestions for improvement of supervision and monitoring.
Was diligent about compliance with Bank requests for information.
Played an active role in the overall policy dialogue for tertiary education, enabling high
quality work and maintaining Mece‘s reputation as one of the premiere agencies for tertiary
education in the region.
Facilitated World Bank analytical work on relevant topics not directly included in Mece2
(State-Guaranteed Student Loan Program, International Scholarship Program, inter alia).
This, along with the implementation of the information system subcomponent, made the
PIU a center for the best policy information available on Chilean tertiary education.
Fluid dialogue with the Ministry of Education included constant feedback on technical issues for
the tertiary education sector as a whole, and not just for Mece2 itself. This active technical
cooperation led to a robust program of Analytic and Advisory Activities.
Some of the events that impacted implementation included:
Initial implementation delays largely due to an unrealistic schedule for the start-up of
activities. This was true both in the Project Implementation Unit and in each TEI with a
FIAC subproject and/or a Performance Agreement.
A 1.5 year project extension. The original closing date was extended from June 30, 2009 to
Dec. 31, 2010 in order to finish implementing the subprojects from the FIAC, achieve
substantial progress in the four pilot Performance Agreements signed, and allow for
discussions regarding a new project (Mece3), particularly in light of the presidential
elections (January 2010).
The Government‘s strict ex-ante fiduciary procedures and high personal penalties for
infractions sometimes led to an ―overly cautious‖ approach to execution of procurement.
When doubt arose about whether national or Bank rules might be in force for a given
transaction, implementers often chose to comply with both sets of procedures. This caused
delays and proved difficult to overcome.
The assessment of implementation should be considered against the backdrop of the rapid
expansion of Chilean tertiary education over the course of Mece2. As figure 1 shows, enrollment
increased by 51% between 2005 and 2010, from 646,000 students to 973,000. According to
Chile‘s Survey of National Socioeconomic Characterization (CASEN for its Spanish-language
acronym), 25% of students enrolled in 2009 hailed from the bottom two income quintiles and
43% from the bottom three quintiles. During this period of massive expansion, Mece2 played a
key role in helping TEIs increase equity while simultaneously improving the quality of their
education. Mece2 did this directly by supporting specific TEI-driven initiatives and indirectly by
prompting others TEIs to take on similar efforts. As this ―spillover‖ effect rippled through the
tertiary education system it further leveraged Mece2‘s investment.
8
Figure 1. Tertiary enrollment in Chile between 1998 and 2010
Source: SIES (http://www.mineduc.cl/index6.php?id_seccion=3224&id_portal=46&id_contenido=13035)
2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization
At Preparation, the main source of information about tertiary education was the voluntary
contribution of data organized through the Council of Rectors of Chilean Universities (CRUCH
in its Spanish language acronym). These institutions, however, had complex political and
economic reasons for wanting to withhold or selectively provide data. As a result, when the
project began, there was no reliable information on even basic descriptive statistics such as
overall enrollment. The situation changed dramatically during implementation, thanks to the
2006 Quality Assurance Law. The Law created the National Observatory for Tertiary Education
(SIES) and made it a legal requirement for institutions to provide accurate data to the
Government. In addition, only TEIs who reported data were eligible for Mece2 funds. By 2009
99% of institutions were complying. The SIES, financed by subcomponent 1.4, vastly increased
the amount and quality of data available throughout the tertiary education system. This would
have been considered impossible in 2005.
The SIES tracks a variety of statistics related to tertiary education and labor market outcomes for
graduates by institution, including duration of study across degree programs, drop-out rates, and
overall enrollment by institution and by degree program. Additionally, labor market outcomes,
including employment rates and expected salary levels at graduation and over time are also
available. Data improvement throughout the life of the Project was incremental and continuous.
As new information became available, it became increasingly apparent that data originally used
for indicator baselines had been poor and unreliable. Today, Chile has one of the best, if not the
best, tertiary education information systems in Latin America. This statistical compendium
allows for straightforward analysis of the impact of changes in policies on high-level metrics.
This is an important step forward for education policy, as regulators and educators can base
future policy on reliable and rigorous information.
9
M&E design
Although the quality of information improved dramatically over the life of the Project, the
PAD did not sufficiently recognize the poor quality of data available at the time of
Preparation. Baselines were in need of constant review and revision as better data became
available. Wider and more accurate figures routinely replaced what came to be seen as the
―best estimates‖ that were included in the PAD.
Indicators did not always tie directly to Project activities. In some cases, the chain of
causality from Project activity to indicator was diffuse or weak.
No indicators tracked outcomes expected in the PAD‘s Economic and Financial Analysis.
M&E Implementation
Data for indicators was frequently collected and continuously updated. To enable better
tracking of performance and progress, the PIU revisited indicator baselines as data improved.
The PIU also broadened the number of indicators tracked to include the outcomes generated
by Mece2 in private universities, technical training centers and professional institutes. This
was important to capture because Mece2 had ―spillover effects‖: TEIs not supported by the
Project copied some of the developments and innovations implemented in TEIs that were
supported.
At the request of the Task Team, the PIU developed quantitative performance metrics across
eight qualitative dimensions of FIAC projects. These metrics improved vetting of outcomes,
and allowed better comparisons of methods and results across projects.
An International Advisory Committee was established with renowned experts to monitor
and evaluate the progress of Mece2 and give advice with regards to its strategic direction. In
mid-2010, members of this Committee conducted the Midterm Review of the four
Performance Agreements and an evaluation of the four triggers that conditioned the second
phase of the Adaptable Program Loan. These assessments were both positive, resulting in
congratulatory remarks and feedback.
M&E Utilization
The data tracked by the SIES is used continuously in policy decisions by the Ministry of
Education and the GoC as a whole. Think tanks draw on the SIES regularly.
Employers, students, and families use more and better quality information. The labor market
observatory has become particularly important for employment and career decisions made
by students and their families.
The SIES serves as the basis for analytical studies on tertiary education, including several of
the reports produced by the World Bank.
2.4 Safeguard and Fiduciary Compliance
As part of the implementation of the IPDP, the PIU sought to gather the best possible
information regarding the effectiveness of existing scholarships and financial aid for indigenous
Chileans in tertiary education. Policies and benefits specifically for indigenous students already
existed in Chile prior to Mece2, but there was very little information regarding their
effectiveness. The Task Team encouraged more accurate measurement and data collection on the
situation of indigenous students early on in the project. This was particularly important given
there was no empirical base for additional indigenous student support mechanisms and no data
regarding the impact of existing policies.
10
As better data became available, the focus of the IPDP became ensuring indigenous Chileans had
access to new remedial opportunities made available under the Project. Initial data indicates that
indigenous students are benefitting from these programs in amounts greater than their proportion
of the overall tertiary student population, with positive impacts on retention and graduation rates.
Further analysis is on-going, and the next project is expected to mainstream the results achieved
for indigenous students. ISR ratings of the implementation of the IPDP were satisfactory.
With regards to fiduciary compliance, the Project encountered some difficulties during
implementation. There was insufficient coordination between the TTL, procurement specialists
and consultants designated by the procurement specialists to carry out fiduciary supervision. This
resulted in a failure to agree on a common approach for supervision and on a realistic set of goals
for mitigating deficiencies during the last two years of implementation. Procurement planning
and compliance moved towards being an end in itself and not a means of appropriately and
efficiently procuring goods and services. The task team failed to develop clear criteria to guide
the TEIs regarding which items to include in their procurement plans. TEIs thus felt obliged to
include all transactions regardless of their size, resulting in unwieldy and ineffective
procurement planning. By 2010 the procurement plan had 2300 entries with an average
transaction size under $10,000.
2.5 Post-completion Operation/Next Phase
Although the four triggering conditions that allowed Mece2 to transition to the second phase of
the APL were met, the GoC has requested a new project. The APL2 is no longer an adequate
instrument for the GoC because of Chile‘s changed policy context brought about by the election
of a new administration in January 2010. Building on the PDO of Mece2, the GoC may seek to
fine-tune the new PDO to better fit the current context. In addition, because the size of the
Mece3 loan ($100 million) is four times that of Mece2‘s ($25 million), the GoC may seek to
broaden the PDO. Mece3 will extend the reach of Mece2 by extending performance-based
funding agreements and specifically and proactively addressing quality and relevance in
pedagogy programs, professional institutes and technical training centers, among others.
Transition arrangements
Since project closing the Government has continued to finance the activities of the PIU, and will
continue to do so until Mece3 starts to operate in January 2012. More importantly, the Ministry
of Education recently absorbed the PIU, renaming it the Department of Institutional Funding and
giving it formal responsibilities for all public funding of TEIs. This type of arrangement is a
positive step forward because it ensures continuity and permanence for Mece2 activities.
Performance indicators
M&E will continue until it is revised during the new project. Because the SIES has become the
information backbone of the tertiary education system, hundreds of indicators are continuing to
be tracked, collected and analyzed.
Follow-up by the Bank
The Bank is expected to deliver Mece3 in time for the GoC to include it in its annual budget.
11
3. Assessment of Outcomes
3.1 Relevance of Objectives, Design and Implementation
In 2010, Chile became a full member of the OECD. OECD member countries have the world‘s
most complex economies, structures of production, and need for highly skilled human capital.
As Chile seeks to achieve mean income levels and other economic and social indicators of its
OECD peers, the Project objectives are arguably more pertinent than ever. The Program
Objective of supporting Chile‘s development into a knowledge-based economy by strengthening
its tertiary education system therefore continues to be highly relevant.
The areas of the PDO continue to be a focal point in the 2011-2016 Country Partnership Strategy.
The CPS outlines a general vision of the country improving competitiveness and achieving
sustainable growth, while reducing inequities and poverty. Stakeholders from civil society and
the private sector identified that a key factor in this vision was improving education quality.
The goals articulated in the CPS directly build upon successes of the Mece2 design. The project
design emphasized performance-based agreements, whereby future disbursements were
contingent upon visible progress against agreed upon metrics. This is a significant departure
from previous disbursement mechanisms that were simply tied to historical allocations and
lacked any incentive or performance aspect. The CPS calls for continued transition from
historical allocations to performance-based agreements. Linking funding allocation to a feedback
loop where disbursements are dependent on progress toward goals and priorities set by the
Ministry of Education further empowers the GoC to articulate clear policy goals across TEIs.
The implementation of the Project is highly relevant to the CPS. The CPS envisions well-
managed TEIs coalescing around goals, and modulating their approaches based on information
as it accrues. Though significant weaknesses remained from the design of the Mece2 PDO and
its related key performance indicators, the Project as implemented has lead to observable and
significant sector-wide improvements. The Project allowed the management of TEIs to become
more professionalized; moreover, through the SIES and through studies funded as part of the
lending program, the Ministry of Education has increased the relevance and coherence of tertiary
education policy across institutions. The data and information systems central to Mece2 have
allowed for a feedback loop, whereby forward decisions are based on outcomes and data trends
visible from prior decisions. In this respect, the Project as implemented has resulted in significant
capacity building. The increased transparency of results and trends from M&E has consolidated
more power with the central management of institutions whereas before each individual faculty
within an institution directed its own policy.
3.2 Achievement of Project Development Objectives (PDO)
Mece2 increased the effectiveness of public funding for tertiary education by enhancing
coherence, responsiveness, quality and equity in the system, through stronger accountability for
performance. Below listed are Mece2‘s specific contributions to each PDO.
Coherence
Led directly to the conversion of the PIU into the Ministry of Education‘s Department of
Institutional Funding within the Division of Higher Education. The mandate of the
Department is to rationalize and optimize the direct public funding of tertiary education
12
institutions. Chile has never had such an explicit mechanism to ensure the coherence of
funding policies. The creation of the Department marks a significant departure from the
―historical‖ basis of funding, and facilitates a central goal of the Project.
Promoted the design and implementation of a credit transfer system (SCT-Chile: www.sct-
chile.cl). It is now possible to transfer credits earned in technical tertiary education toward a
university degree within a single institution, and implementation has begun to enable
transfers between institutions as well. Lack of ―pathways‖ from technical to university
education was a perennial problem before, and this is a major accomplishment. That Mece2
vastly exceeded the student transfer indicator testifies to the success of this initiative.
PDO Indicator # 1 Baseline Target End value
Number of students graduating from CFTs
and IPs transferring to universities 72 (revised) 1000 2,432
Promoted the creation of 18 Institutional Research Units and 22 Teaching and Learning
units in TEIs. These units have increased the coherence of institutional strategies at their
home institutions.
Implemented four performance agreements and succeeded in transforming institutional
culture at the pilot institutions, showing a clear way forward in tertiary education. By
linking future funding to realization of the Ministry of Education‘s policy goals, the
coherence of national tertiary education will continue to improve.
Intermediate Outcome Indicator # 12 Baseline Target End value
Number of universities covered by a
performance agreement 0 4 4
Developed seven major policy studies. These continue to be used to strengthen and guide
tertiary education policy. Studies examined causes of first-year drop-out; university leaders'
perception of Divesup policy instruments; employers' and graduates' perceptions of the
quality of higher education graduates; the situation of advanced human capital for education,
research and innovation; the current state of the higher education sector; the newly
implemented foreign scholarship program; and the situation of indigenous students. These
studies are of high interest to National Educational policy and demand for them exceeded
the goals initially set by Mece2.
Intermediate Outcome Indicator # 2 Baseline Target End value
Number of studies on Tertiary Education
conducted by date 0 4 7
Responsiveness
Set a new standard for responsiveness in education financing through four performance
agreements with Cruch universities. These resulted in a high attainment of project goals,
with the majority of agreed upon metrics reached.
Targeted the competitively allocated FIAC to tactically address the needs of institutions in
specific areas, in alignment with the priorities of national education policy. These areas
centered around improving academic instruction and management quality, specifically
focusing on advancing human capital, academic innovation, improvement of academic
management, and curriculum reform. The competitively allocated funds were responsive to
the needs of both the institutions as well as the Ministry of Education. Since no single
13
institution could receive more than 18% of FIAC funds in any single round, funds were
equitably distributed regionally and among TEIs. The target for subprojects was exceeded.
Intermediate Outcome Indicator # 6 Baseline Target End value
Number of sub-projects supported at the
tertiary level (cumulative) 30 120 371
The PIU, at the suggestion of the Bank, developed a quantitative assessment system for
measuring the quality of output from FIAC subprojects. The 371 subprojects were reviewed
at mid-term and completion on 8 dimensions of quality. Individual grades were given for
each dimension, and discussed with subproject teams. Almost 80% of subprojects were
rated at or above satisfactory. Special ―probationary‖ attention was given to low-scoring
projects, and unsatisfactory projects could be terminated less improvements were made.
Number of projects % of total Grade
26 7% excellent
265 71% good / satisfactory
53 14% marginally satisfactory
27 7% unsatisfactory
Quality
Financed the implementation and expansion of a quality assurance system such that by the
end of the Project accreditation was accepted as the norm for all TEIs. In 2006 Congress
passed the Quality Assurance Law and the National Accreditation Commission became the
principal agency for quality assurance in Chile. Accreditation targets were exceeded.
Intermediate Outcome Indicator # 3, 4 and 5 Baseline Target End value
Percentage of tertiary students in accredited programs and programs in the process of being accredited.
Undergraduate 31% 37% 51%
Master‘s 12% 20% 35%
PhDs 22% 60% 85%
Promoted an increase in the quality of pedagogy programs, the number of doctoral programs
and the percentage of faculty with doctoral degrees. The latter increased to 40.9%, from a
baseline of 33%. Directly through Mece2 funds, 69 faculty members are getting PhDs, 15
are pursuing post-doctorates, and 170 have benefitted from short specialization programs
abroad. Additionally, 347 students have PhD scholarships and 5 have graduated to join the
ranks of faculties. Today, there are 4,055 PhD students in Chile. These developments have
raised the quality of tertiary instruction across the board, and will continue to do so, as
institutions work to retain their accreditation.
Intermediate Outcome Indicator # 10 Baseline Target End value
Number of enrolled doctoral students in Chile 2,334 (revised) 2,500 4,055
Promoted increases in retention rates and a 2.5 month average decrease in time-to-degree.
Gave each of the four TEI performance agreement pilots a chance to define what improved
quality meant, and receive funding to meet agreed indicators. The fact that the majority of
agreed indicators were met argues that the Agreements were significant promoters of quality.
14
Equity
During the course of the project, the number of Chilean tertiary students from the bottom two
income quintiles (Q1 and Q2) rose from 125,000 to 219,000. This equals 29% of all new
students for the period. Since by definition Q1 and Q2 represent 40% of the Chilean population
(each quintile is a fifth of the population), the enrollment rate is converging on the proportion of
Chilean families at this income level. This represents massive progress against earlier times
when students from upper quintiles were enrolling at ten times the rate of poorer students.
These new Q1 and Q2 students were almost exclusively ―first-generation‖ and many were from
poor quality secondary schools. The objective of increasing equity in tertiary education was met
through funding projects to strengthen students‘ basic competencies and increase teacher quality.
The number of students in remediation programs reached over 35,713, surpassing the target
many times over. Chilean TEIs now take responsibility for making sure admitted students
succeed in ways they did not prior to the Project.
Intermediate Outcome Indicator # 8 Baseline Target End value
Enrollment in preparatory programs 307 (revised) 3,900 35,713
The number of teachers with post-graduate degrees increased significantly over the span of
Mece2, along with the number of students enrolled in post-graduate education. Over time, this
should increase the quality of faculties and thus should promote greater achievement across
student backgrounds and learning styles. Projects to strengthen basic competencies were tailored
to reduce the impact of poor quality secondary education instruction, and increase achievement
of tertiary enrollees with more limited basic skills. The massive increase in Q1 and Q2
enrollment rates changed the composition of first year students: the latter contained an increasing
proportion of students with lower chances of succeeding and higher expected drop out. The first-
year drop-out rate, which remained stable around 21% for Cruch universities and decreased by 2-
3% in private universities, IPs and CFTs, is a testament to the efficacy of these program
enhancements. Overall, the proportion of students from the lowest socioeconomic quintiles in
tertiary education increased from 21% in 2003, to 23% in 2006, to 25% in 2009.
3.3 Efficiency
The value of the Project has been assessed through two distinct NPV calculations, each focusing
on an outcome associated with Mece2. The approach is conceptually similar, if not the same, as
that originally used in the Project Appraisal Document (PAD). As mentioned before, however,
none of the indicators defined in the PAD tracked the outcomes expected in the PAD‘s economic
and financial analysis. The data for the below analysis comes directly from SIES.
Estimates indicate Mece2 has an NPV USD (2005) of $677 million. The funds associated with
the Project had an NPV USD (2005) of $80 million, suggesting that conservatively, the total
leverage of Mece2 was approximately 8.5. This conclusion is explained in detail in Annex 3:
Mece2 has increased tertiary education quality, leading to an increased premium for wage
earners with tertiary education. Thirty percent of this benefit was assumed to be directly
attributable to the Project and its value is estimated at NPV USD (2005) $342 million.
15
Mece2 has led to shorter-duration academic programs and quicker time-to-degree, resulting
in time savings and increased career earnings for graduates of tertiary education. This
benefit is estimated at NPV USD (2005) of $335 million.2
As per the PAD, Mece2 is also expected to result in additional value by increasing
graduation rates; this effect (and its value) cannot yet be observed. Because Mece2 targeted
its efforts to increase retention rates on first year tertiary education students, beneficiaries
have not yet graduated.
3.4 Justification of Overall Outcome Rating
Rating: Satisfactory
The Project is rated satisfactory. Through stronger accountability for performance, important
strides have been made in increasing the coherence, responsiveness, equity, and quality in the
system. Major successes associated with the Project include:
The legislation on quality assurance and student financial aid, and the subsequent major
increases in equity, enrollment, and accreditation across Chile.
The robust and continuously updated Tertiary Education Information System (SIES).
The performance agreements piloted with four universities, including proof-of-concept for a
scale-up of results-based financing.
The increases in advanced human capital and research capacity, instrumental for a
knowledge economy.
3.5 Overarching Themes, Other Outcomes and Impacts
(a) Poverty Impacts, Gender Aspects, and Social Development Due to inadequate preparation during secondary education, a notable group of undergraduates
enrolling in tertiary education today lack basic skills and require remedial work prior to
embarking on degree programs. Historically, in the absence of remedial or basic competencies
opportunities, these students faced high drop-out rates and/or slow educational progress.
A number of elements of the Mece2 program directly addressed this source of inequity. Several
institutions receiving funds through the performance agreements established or expanded
remedial classes. One of the loan triggers targeted a minimum enrollment of 2,000 students in
remedial courses, and by the end of 2008 had attained actual enrollments of 9,724 students. By
2010, 35,713 students had enrolled in such classes; while 75% of them were in programs
supported directly by Mece2, the remainder was in independently created programs. Indeed,
Mece2‘s initial push for remediation courses created a powerful spillover effect. The expanded
remedial education likely accounts for a portion of the reduction in drop-out rates in the lower
socioeconomic quintile undergraduates.
2 The granularity of the data was complete only to the university level. Analyzing just the subset of
CRUCH universities instead of all the universities (Cruch & private) would have been preferable,
however, as most of the funds associated with Mece2 accrued to CRUCH universities. This may make the
NPV more conservative, as CRUCH degree durations are expected to have decreased more than the
university-wide average.
16
Additionally, the competitively allocated FIAC targeted projects to increase the effectiveness of
the remedial curriculum, and included funds directed at increasing educator quality through
improved pedagogy curriculum and increased numbers of pedagogy graduates. Over time,
increased rigor in teacher education is expected to increase teacher quality. Ultimately this is
expected to drive significant social development as higher quality teachers return to their
communities and increase the quality of other educational institutions.
In terms of gender, female undergraduate enrollment in tertiary education has increased from
48% to 51% of total enrollment between 2005 and 2010. In addition, females have higher
retention rates than males in their first year of study (72% versus 66%), and have an average time
to degree that is two semesters shorter than males (10.5 semesters versus 11.6 semesters). These
changes may lead to increased female participation in the labor force.
(b) Institutional Change/Strengthening
Mece2 has driven significant change in tertiary education through strengthening information
infrastructure, and enhancing policy direction and accountability in management.
The SIES has allowed for significant institutional strengthening by creating a policy feedback
loop based on outcomes. In particular, the quantity of funds allocated through performance
agreements are dependent on measurable progress towards goals agreed to by the institutions and
the Ministry of Education. More transparent outcomes have increased accountability in the
system, and with results closely tracked and measured for improvement, institutions have an
incentive to develop strong coordinated policies to improve performance. This has had the effect
of increasing the centralization of policy and power with institutional management.
Importantly, Mece2‘s Project Implementation Unit has been fully institutionalized in the
Ministry of Education as the entity responsible all funding to TEIs that is not student-related.
Initial conversations with the GoC suggest that more of the direct transfer funding for CRUCH
universities will evolve to performance based funding. Additionally, if educational funding
continues to be allocated through performance agreements, as is expected in the upcoming
Mece3 project, the increased accountability of the funding mechanism will allow the Ministry of
Education to coordinate national educational policy more closely with the newly strengthened
institutional management. Previously, with power decentralized across various faculties, this
coordination would not have been possible.
(c) Other Unintended Outcomes and Impacts (positive or negative) Mece2‘s reputation allowed it to facilitate analytical reports on tertiary education quality beyond
the number expected. The World Bank and the OECD collaborated on two major studies—one
on the tertiary education system and one on Chile‘s Foreign Scholarship Program (Becas Chile).
The recommendations of both reports were well received and key elements adopted.
The success of the GoC in implementing projects to bolster the quality of tertiary education has
resulted in significant South-South knowledge transfer. Chile has shared its experience and
expertise in implementing or advising funding programs in Tunisia, Bangladesh, India, Ghana,
Mozambique, Lesotho, Montenegro, and Indonesia.
17
Finally, Mece2‘s increased focus on tertiary education corresponded with an increased
commitment by the GoC to improving equity in tertiary education, manifest most visibly in the
new State-Guaranteed Student Loan Program (CAE). The financial commitment to CAE has
dwarfed the scale of the commitment to Mece2 activities, with ongoing annual disbursements at
a scale similar to the entire lifetime disbursements of the Mece2 project.
3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops
No beneficiary survey and/or stakeholder workshops were conducted for the ICR.
4. Assessment of Risk to Development Outcome
Rating: Negligible to low
Despite the March 2010 change in administration (from President Bachelet to President Piñera),
the Government has focused time and effort on improving quality and equity in tertiary
education. Going forward, the GoC will emphasize incentive schemes for students, educators,
and institutions to improve management and results based on standards, and will promote equity
through scholarships and loans. The GoC is also developing training programs for educators and
better information systems for graduates to have access to more granular labor market data.
Many of these goals will be addressed through the Mece3 project. The risk to development
outcomes is thus negligible: the Government has made specific policy decisions to further and
continue the objectives of Mece2, and has requested preparation of Mece3 to extend their reach.
Enhance quality and relevance of tertiary education
This will likely be one of the development objectives of the new Mece3 project. The latter‘s size,
$100 million, is a reflection of the Government‘s commitment to this issue. The new project will
extend the reach of Mece2 by specifically and proactively addressing quality and relevance in
technical education (professional institutes and technical training centers).
Ensure access to tertiary education for all talented individuals
Equity in Chile has been significantly enhanced through programs complimentary to Mece2. The
CAE program gives academically qualified, poor students loans to attend tertiary education.
CAE has grown exponentially and now has over 216,000 borrowers (equivalent to almost one
fourth of Chile‘s undergraduate students) and 63% of them come from the lowest two income
quintiles. The Government has taken steps to improve implementation and increase repayment
rates among borrowers, and the program is highly likely to continue.
Improve internal efficiency and strengthening accountability for results
The Division of Higher Education in the Ministry of Education continues to upgrade the quality
of its human capital and improve its internal efficiency. Several new PhDs have been brought on
board, including one in charge of managing the newly created the Center for Higher Education
Studies within the Ministry. This center will further improve internal efficiency by providing
opportune evidence and information for policy decisions.
In terms of accountability for results, the Government has requested that all the resources in the
new project be disbursed using competitively-allocated, performance-based contracts. In addition,
18
the Government wants to change the way base funding is allocated to public universities and the
President and Minister of Education have already discussed a proposal to move towards a
performance-based system for an additional 5% of base funding.
5. Assessment of Bank and Borrower Performance
5.1 Bank Performance
(a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory
The PAD provided high quality guidance and presented a strong case for investment. The
strategic relevance and approach of Mece2 was clear within the specific country context, and the
technical, financial and economic aspects of the lending program evidenced rigorous and
thorough analysis. Moderate shortcomings included:
The indicator structure was overly defined and had limited regard for sufficient data
availability and quality.
The link between APL Program Objectives, PDO and key performance indicators was
unclear. The APL Program Objectives were phrased in such a way that they did not easily
align with the statement of the PDO. Moreover, the Key Performance Indicators were not
clearly linked to the specific Program/PDO objectives.
No indicators tracked outcomes expected by the PAD‘s Economic and Financial Analysis.
The timeline for the Project did not sufficiently take into account the Borrower‘s regular
cycle for budgetary approval in Congress.
Project implementation arrangements significantly increased transaction costs. Since the
majority of items in the Procurement Plan were non-procurable, fiduciary responsibilities
would have been better carried out through Financial Management rather than through
procurement mechanisms. Insufficient coordination on this front created duplicative work.
(b) Quality of Supervision Rating: Moderately Satisfactory
The Bank was particularly strong with regards to its focus on development impact, its technical
supervision, and its role in ensuring and encouraging adequate transition arrangements after
project closing. In addition to timely compliance with normal supervision responsibilities, the
Bank suggested new ways of measuring outcomes of FIAC subprojects. The Bank Task Team
maintained active communication with the PIU. The TTL was regularly invited to address
important tertiary education events, and his opinion was routinely sought by the Chilean press on
related issues. Fluid dialogue with the Ministry of Education included constant feedback on
technical issues for the tertiary education sector as a whole, and not just for Mece2 itself. This
active technical cooperation led to a robust program of Analytic and Advisory Activities.
Supervision of safeguards aspects was active and vigilant, assessing the Project regularly.
For a satisfactory rating, procurement difficulties would have been addressed sooner in the life of
the Project. Insufficient coordination between the TTL and the procurement specialists resulted
in a failure to agree on a common approach for supervision and on a realistic set of goals for
19
mitigating deficiencies during the last two years of implementation. Procurement planning and
compliance thus moved towards being an end in itself and not a means of appropriately and
efficiently procuring goods and services.
(c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory
Overall Bank performance in ensuring quality at entry and in the quality of its supervision was
moderately satisfactory. The PAD provided effective guidance and presented a strong case for
investment. It would have benefitted from a more flexible indicator structure, a better selection
of indicators, a more streamlined procurement plan, and a more realistic timeline for
implementation. Supervision was very strong on technical aspects and on post-closure transition
arrangements, but would have benefitted from more in-the-field coordination between the core
task team and the fiduciary team.
5.2 Borrower Performance
(a) Government Performance Rating: Highly Satisfactory
The GoC showed strong commitment to achieving project development objectives, facilitating
the enabling environment for Mece2, utilizing the data generated through it in its decision-
making, and ensuring adequate transition arrangements for it after closing. The GoC:
Passed the 2006 Quality Assurance Law and widely encouraged institutional accreditation.
Passed the 2005 Tertiary Education Student Financing Law and extensively financed and
promoted the expansion of the student aid system.
Commissioned several additional tertiary education policy studies, including a WB-OECD
report on the BioBio Region, and a WB report on the state-guaranteed student loan program.
Increased the breath of the tertiary education information system and actively used its data.
Integrated the PIU into the Ministry of Education and renamed it the Department of
Institutional Funding, making it fully responsible for all public funding for TEIs.
Transitioned Mece2 into a new project where resources will be conditioned on performance.
Financed the time gap between the Mece2 and Mece3 projects.
Is currently discussing how a portion of traditional funding for public universities can be
linked to performance.
(b) Implementing Agency or Agencies Performance Rating: Highly Satisfactory
The Project Implementation Unit (PIU) showed strong commitment to achieving project
development objectives, conducted frequent formal and informal stakeholder consultations, put
in place adequate monitoring and evaluation arrangements, and developed solid relationships
with stakeholders and partners. More concretely, the PIU:
Improved the indicator structure originally set out in the PAD, re-setting baselines when
new information was available and increasing the number of indicators tracked to more
adequately inform decision-making.
20
Facilitated additional work requested by the Government to the World Bank.
Compensated for weak institutional capacity in the Ministry of Education‘s Division of
Higher Education, particularly in the initial years of the project. In later years, and more
recently, it became a conduit for extra capacity.
Maintained fluid communications with Government and TEIs, quickly becoming a conduit
between the two. Proof of this is the PIU‘s recent absorption into the Ministry of Education,
and its new additional responsibilities regarding all public funding for TEIs.
(c) Justification of Rating for Overall Borrower Performance Rating: Highly Satisfactory
The Borrower showed strong commitment to achieving project development objectives: it
facilitated the enabling environment for Mece2, put in place adequate monitoring and evaluation
arrangements, conducted frequent formal and informal stakeholder consultations, and ensured
adequate transition arrangements for Mece2 after closing. Even with the initial project
implementation delays due to the Borrower‘s regular cycle for budgetary approval in Congress,
the Borrower‘s performance can be assessed as highly satisfactory.
6. Lessons Learned
Mece1 and Mece2 both provided valuable learning opportunities regarding tertiary education
reforms, leaving lessons that could prove useful for projects both in Chile and in other country
and regional contexts. The primary take-away from Mece2 is the importance of having a well-
defined results framework from the beginning and ensuring the indicators in the framework are
explicitly related to the PDO. In addition, a key lesson is taking advantage of the opportunities
afforded by coincidental timing—in this case, supportive legislation being enacted at just the
moment when it could be leveraged to achieve program objectives. Finally, the following
summarize specific lessons gleaned through the course of project implementation and assessment.
Competent and stable leadership and team in the PIU were a key component of Project success.
The PIU already had institutional experience with the Mece1 project, which facilitated
implementation of Mece2. More importantly, however, the team was composed of competent
and well-prepared professionals committed to Project goals. Little turnover allowed for
continuity and good leadership facilitated Project coordination, execution and credibility.
Rigorous results scoring in FIAC subprojects increased comparability and performance tracking.
To understand which FIAC subprojects functioned well and which did not, performance metrics
across eight qualitative dimensions were developed for projects that did not easily lend
themselves to explicitly quantitative assessments. These metrics allowed for better vetting of
outcomes, and also served to compare methods and results. Without a rigorous approach, the
default tendency was to assert all subprojects had worked well or poorly, with no differentiation.
Realistic timelines for implementation require taking into account the Borrower‘s cycle for
budgetary approval in Congress and the internal approval processes for the various legal
documents needed throughout the life of the Project. For both Performance Agreements and
FIAC, resources awarded in one fiscal year could only be disbursed the following year, initially
21
triggering some implementation delays. In addition, because the time required for internal
approval processes is highly variable, adequate flexibility should be built into Project timelines.
Execution of performance-based funding requires experienced staff to mediate between the
universities implementing the agreements and the Ministry of Education. As implemented, the
performance agreements depended heavily on negotiators hired specifically to mediate between
the TEI Presidents and the Ministry of Education. Without suitable negotiators, national policy
goals would be difficult to reconcile with the reality of the institutional environments where they
were meant to be implemented. The highly qualified negotiators helped the agreements gain
traction. Generally these were former Presidents or academic Vice-Presidents of universities
themselves, familiar with and well regarded in the academic community. Their past experience
gave them credibility with both parties to the negotiations, and allowed the negotiations of
performance metrics to be productive and grounded in the reality of the situation of each
institution. Further, these negotiators were instrumental in developing specific goals for each
university and the detailed indicators used to measure progress in achieving them.
7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners
(a) Borrower/implementing agencies
The Borrower noted this ICR accurately represented the primary activities completed by the
Project, the results obtained and the impact made. Specific comments were incorporated
throughout the text.
The Borrower‘s own Implementation Completion Report is attached in Annex 7.
(b) Cofinanciers Not applicable.
(c) Other partners and stakeholders Not applicable.
22
Annex 1. Project Costs and Financing
(a) Project Cost by Component (in USD Million equivalent):
Components Appraisal Estimate
(USD millions)
Actual/Latest
Estimate (USD
millions)
Percentage of
Appraisal
Strengthening institutional
framework for tertiary education 8.00 8.00 100%
Sub-projects to enhance
coherence, efficiency, equity and
quality
83.25 83.25 100%
Unallocated 0.75 0.75 100%
Total Baseline Cost 91.25 91.25 100%
Physical Contingencies
0.00
0.00
100%
Price Contingencies
0.00
0.00 100%
Total Project Costs 91.25 91.25 100%
Front-end fee PPF 0.00 0.00 100%
Front-end fee IBRD 0.13 0.13 100%
Total Financing Required 92.13 92.13 100%
(b) Financing
Source of Funds Type of
Cofinancing
Appraisal
Estimate
(USD
millions)
Actual/Latest
Estimate
(USD
millions)
Percentage of
Appraisal
Borrower 75.00 75.00 100%
International Bank for
Reconstruction and Development 25.13 25.13 100%
23
Annex 2. Outputs by Component
The project goal of increasing the effectiveness of public financing for tertiary education was
accomplished by way of two separate components with multiple sub-parts. Component one
focused on strengthening the institutional framework for tertiary education at the policy level
through government capacity building, while component two comprised sub-projects, either
competitively allocated or based on performance agreements, that were implemented across a
wide number of tertiary institutions and across a number of focus areas.
Component One: Strengthening the Institutional Framework for Tertiary Education
Subcomponent 1.1 - Improvement of Sector Oversight and Regulation
To increase sector oversight, funds were used to create an office within the Ministry of
Education to focus on tertiary education policy. In part, personnel hired for this office provided
support for the implementation of a 2005 law on student financial assistance. The law gave rise
to the state-guaranteed student loan program for tertiary education, which is now providing
funding for over one hundred thousand new students each year. Additionally, funds were used to
hire staff to design and implement the performance agreements. Conceptualization and program
design for the competitively allocated funds was also financed by this subcomponent.
Subcomponent 1.2 - Strengthening of Policies and Strategies for Tertiary Education
The Government of Chile commissioned a number of studies to strengthen tertiary education
strategies and policies. The goal of four studies was exceeded by three. The studies focused on
critical policy areas specified by the Ministry of Education. Their translated titles are:
Study to determine causes of dropout in the first year of college.
University leaders' Perception of the Policy Instruments of the Division of Higher
Education in the Ministry of Education.
Employers' and Graduates' Perceptions Regarding the Quality of Higher Education
Graduates.
Survey and Evaluation of Advanced Human Capital for Education, Research and
Innovation in Chile.
Reviews of National Policies for Education: Tertiary Education in Chile, co-authored by
the OECD and The World Bank, 2009.
Human Capital Formation Abroad: A Review of Chile‘s Higher Education Scholarships
Programme, co-author by OECD and the World Bank Review, June 2010.
Description and Evaluation of Educational Policies Regarding Access to Benefits of
Indigenous Students in Chile.
Subcomponent 1.3. - Consolidation of the National System for Quality Assurance
In 2006, law 20,129 was passed, creating the National Accreditation Commission, which has
coordinated accreditation of TEIs, and set quality and process standards across tertiary education.
Project funds, subsequent to the passage of the law, were used to fund infrastructure, consultant
services, and operational expenditures involved in the establishment of the commission. Funds
have been used to understand elements common to the highest level international quality
assurance programs. Further, Mece2 facilitated study of common elements in provision of
24
quality tertiary instruction across Chile. The number of accredited institutions has doubled from
the end of 2005 to 2010, rising from 41 to 87.
Subcomponent 1.4 - Establishment of a National Observatory for Tertiary Education
The law that established the national quality assurance commission also created a system for
information in higher education. Participation in the information system was made compulsory
for accredited institutions. Institutions participating in the performance agreements and the
competitively allocated FIAC were required to share indicator information with the observatory.
The system collects information at the level of enrolled students, and then aggregates it for
public access.
The information system has allowed for unprecedented transparency into tertiary education, with
information on historical enrollment, graduation rates, faculty characteristics, program durations,
as well as information regarding labor market outcomes by study area. Labor market outcomes
include current wage data, historical wage trajectory data, unemployment percentages, as well as
the underlying cost of study. Aggregated data is publicly available through a user-friendly
website.
Subcomponent 1.5 - Project Management
Project management funds facilitated all operations associated with performance agreements and
the FIAC, which are described further below. These funds expanded the Ministry of Education‘s
capacity to coordinate with representatives from tertiary institutions, allowing them to better
facilitate compliance with program requirements. Funds were used to design and execute
evaluation metrics for projects in the competitively allocated FIAC. Additionally, funds were
used to hire appropriate representatives for negotiation of the performance agreement indicators
and objectives, as well as international experts for evaluation of program success.
Component Two: Sub-Projects for Enhancing Coherence, Efficiency, Equity and Quality
Subcomponent 2.1 - Competitively Allocated Academic Innovation Fund (FIAC)
The FIAC funds were allocated to 371 distinct projects in five critical policy areas. FIAC funds
were matched at 30% of the funding level with the resources of the implementing institutions
through their own internal resources. The projects were rated across eight factors, including
integration with the university‘s strategic plan, implementation of the project, meeting project
deadlines, accomplishment of hoped-for results, achievement of project indicators, quality of
management, replicability of the project, and commitment of the institution to the project. In
general, projects scored highly, with most projects rated satisfactory. The lagging grade on the
majority of projects was the achievement of the agreed upon indicators. General descriptions of
project areas are related below, along with enumerations of example projects.
Development of Academic Personnel for Research and Management
Approximately 20% of the FIAC funds were allocated to projects to increase the number and
quality of personnel with research and/or management capabilities. A variety of projects were
undertaken, with 62 distinct projects implemented across an array of universities. Examples of
projects intended to develop research capabilities include exchange programs for faculty at
25
Pontificia Universidad Católica de Chile, as well as the recruitment of international speakers and
researchers. At the same university, to make research more effective across the region, the
agronomy and biology faculties instituted a program to make research equipment more
accessible to other nearby researchers. At Universidad Santiago de Chile, a biotech initiative was
launched at the post-graduate level. At Universidad de Concepción, resources were made
available to increase the number of faculty members with graduate degrees and to increase
English speaking skills of faculty members. Additionally, the oceanography program was
expanded, with funds used to procure an exploration boat with advanced research equipment.
The program was awarded a seven year accreditation. The Universidad de Valparaiso
strengthened its post-graduate research capacity by recruiting highly qualified individuals to the
university and implementing performance standards for them, resulting in the accreditation of its
graduate degrees. Its neuroscience PhD program is now internationally recognized after
investments were made in major scientific equipment.
Examples of projects focused on increasing management capacity included one in which the
Pontificia Universidad Católica de Chile developed a centralized clearinghouse for faculty to
design and propose FIAC projects, ensuring that the best developed ideas were promoted on
behalf of the whole university. Further, university management ensured that the proposed
projects were coordinated to increase the university‘s capabilities where they were most needed
first. At Pontificia Universidad Católica de Valparaíso, an information management system for
libraries was implemented, and the institutional analysis unit received funding to purchase
software and computers necessary to fulfill its mission.
Development of National Post-graduate Programs
Approximately 35% of the disbursed funds through the FIAC went to create or further develop
post-graduate programs, including fostering collaborative efforts between researchers across
institutions. More than eighty individual projects were undertaken to further develop post-
graduate programs. Examples include strengthening PhD programs at Pontificia Universidad
Católica de Chile through introduction of student scholarships. At the same university, a holistic
approach to postdoctoral research has cultivated a collaborative approach to research in
agronomy and biology, with coordination across both fields. At Pontificia Universidad Católica
de Valparaíso, two joint PhD programs were developed with Universidad Técnica Federico
Santa María. The two coordinated programs were able to attract professors and students, and
gain maximal leverage from investments in equipment. At Universidad Santiago de Chile, the
university partnered with Universidad de Chile to create a joint Microbiology PhD program.
Additionally, a PhD program in history was created. Universidad de Tarapacá created a PhD
program in anthropology. The Universidad de Concepción developed an ecology PhD program
with the Universidad Austral.
Improvement of Learning Outcomes
Approximately 13% of the FIAC funds were disbursed to institutions to improve the learning
outcomes of students through better infrastructure, greater focus on teacher ability, increased
remedial education resources, and closer tracking of student successes and failures. More than
seventy distinct projects were funded across a wide array of universities. For example, the
26
Universidad de Chile increased focus on learning outcomes by linking professor evaluations to
teaching ability in addition to the traditional focal area of research. At Universidad Técnica
Federico Santa María, a Center for Integrated Learning in the Sciences was created to help
students in basic skills and competencies, including bolstering introductory level science skills,
but also teaching time management, study habits, and general planning. Additionally, a system to
track student academic performance was implemented. Teacher accountability was emphasized,
and awards for teacher quality were introduced. At Universidad Católica de Temuco, a teacher
excellence project was implemented to define and publicize what makes a good teacher.
Additionally, an information tracking system for students was implemented to closely evaluate
competences at the start of their university careers. The Universidad de Talca created an alumni
career center to better understand graduates‘ labor market outcomes, increase brand recognition
for the university, and cultivate relationships with potential employers to create more internship
opportunities for its students.
Curriculum Modernization Based on Learning Outcomes and Competencies
Approximately 22% of the FIAC funds were allocated to 119 projects intended to upgrade
curriculum based on learning outcomes. For example, at Universidad de Chile, general education
courses were implemented to increase interactions within the student body for students across a
variety of study areas. Additionally, the university partnered with other local universities to
allow Philosophy students to attend classes at the other institutions. Critically, a program was
undertaken among several universities to increase the transferability of credits across institutions.
Universidad Católica de Temuco instituted an action plan for improvement of undergraduate
education. A Center for Improvement in Teaching was launched in 2008 to define competencies,
and drive curriculum redesign. A program for insertion into university life was launched to help
students successfully bridge the gap between their secondary and tertiary educations.
Improving the Offerings of Higher Technical Education
Approximately 10% of the total funds were allocated to improving the course offerings of
technical education centers. Thirty-five different projects were undertaken, but successes were
limited in this focus area. A number of operational hurdles limited participation, and no funds
were disbursed in several rounds of the allocation competition. High turnover in faculty and
procurement challenges were cited as the main drivers for limited accomplishments. However,
some notable successes included a transformation of curriculum at DUOC to focus on
competencies. Additionally, the number of e-learning offerings was increased at DUOC, and
additional programs in high employment areas were developed. At Universidad Católica del
Norte, a technical degree for ―Tourism Technician‖ was created, a field with high levels of post-
graduate employment.
Subcomponent 2.2 - Performance Agreements
Four performance agreements were established at four different universities. The goals, metrics
and outputs associated with each performance agreement varied by institution. In general, across
the institutions, the performance agreements succeeded in coalescing distinct faculty segments
around common goals as articulated and agreed upon by central management figures. Previously,
the Ministry of Education was unable to engage in policy discussions with a central figure
27
capable of directing change; at the four universities with performance agreements, that situation
has changed.
University of La Frontera
Resources of $2.3M were allocated to the performance agreement with University of La Frontera,
with an additional $1.0M in funding provided by the university. These resources were targeted
toward increasing the capability of management at the university, increasing the quality of
instruction, and increasing the scientific accomplishments of the university in the area of
bioresources and agricultural resources.
In the third year of the agreement, the university had succeeded in meeting a number of the
promised indicators to improve educational quality. A strategy was formulated and implemented
to expand basic core competencies. Remedial education courses were developed for 74% of core
subjects, but the number of students who failed to pass their courses stubbornly stuck at a high
61%. A professional development center was established at the university to track professional
certifications and competencies. An employability test for graduates fell short of the goal of 87%,
but did increase to 81%. The Division of Quality Assurance and Institutional Development was
redesigned, and computer systems and web platforms were put in place to assist in implementing
new policies
The university met a number of the promised indicators for increased scientific output. A new
scientific laboratory was constructed, and new academics with doctoral degrees were hired onto
the staff. Among Chilean universities, La Fontera was ranked seventh in the research and
technological development of bioresources.3 Seventeen patents were requested, significantly
exceeding the goal of seven. Increases in scholarly work led to 90 new publications in ISI,
exceeding the goal of 60.
The university also committed to increasing credit recovery from graduates. The university
exceeded the promised level of ~$2.4M with ~$2.6M in recoveries. To achieve this, La Frontera
developed and implemented the first system among Chilean universities to allow for electronic
payment of tuition, fees, and debts.
University of Bío-Bío
Resources of $2.3M were allocated to the performance agreement with the University of Bío-Bío,
with an additional $0.95M in funds provided by the university. These resources were used to
improve the level of professional and academic success of the students in the university.
The university was successful in reaching a number of the promised indicators to increase
retention and academic quality. The university implemented a number of new programs to
welcome matriculating students, and increase retention of those students. These included an
introduction program for new students, a survey to better characterize the core academic
competencies of incoming students, and a tutoring program for students in mathematics and
3 According to a recent Elsevier Scopus overall productivity review for 2004-2010, Universidad de la
Frontera is now ranked 9 in Chile.
28
engineering. Additionally, leveling programs to ensure basic competencies in incoming students
were developed based on survey results. First-year drop-out rates fell from 17% to 3.5%, and
third year drop-outs have fallen from 40% to 34%. Concern for meeting the metrics of the
performance agreement was cited as a major reason for the decline in drop-out rates, as faculty
focused more time and attention on student retention than they otherwise would have.
The university succeeded in reducing the real duration of its academic programs, with four year
programs declining from a real duration of 6.56 years to 6.25 years, five year programs declining
from a real duration of 7.11 years to 6.75 years, and six year programs declining from a real
duration of 8.5 years to 7.7 years. This represents significant progress toward reducing the time
and cost of a university education.
A program to manage alumni data was implemented to allow the university to maintain linkages
with graduates. The university succeeded in increasing the quality of the integration of its
graduates into the workforce, with an employer satisfaction survey increasing to 95.3% from
94.9%. The number of graduates attaining authority positions increased from 16.7% to 21%, but
failed to reach the goal of 25%. Student satisfaction with course quality actually declined slightly,
from 82.9% to 81.5%, though this decline can be explained by the inclusion of more programs
with lower initial satisfaction levels than were included in the baseline.
The university developed a plan to standardize and manage the accreditation process. The
university succeeded in expanding its accredited program offering, increasing them from 25.7%
to 65%, with the proportion of students in accredited courses increasing to 58.7%, well above the
target goal of 55%. The university created a quality assurance program that, through the
repetition and standardization of processes, has significantly decreased the timeline to earn
accreditation for a program, from one year and four months to just nine months.
University of Chile
Resources of $3.4M were allocated to the performance agreement with the University of Chile
with an additional $0.44M in funds provided by the university. These resources were used to
reorganize central agencies to streamline the management processes of the university, to increase
central management and execution capacity across campuses, and to implement an information
system to support efficient management of the university.
The university management reorganization was composed of three parts; evaluation, re-design,
and implementation. Of the thirty-six organizational units on campus, all were evaluated in the
context of increasing their operational effectiveness and increasing coordination with
management. Half of those organizational units have been re-designed, with some initial
progress in implementing the management redesign.
Limited progress was made in increasing central management and execution capacity across
campuses. A new administrator was hired for the South Campus as part of the expansion of
central management, but no additional progress was made.
29
The University succeeded in implementing a new integrated information system that allows easy
access to critical information, including classroom facilities, library archives, human resources
information, teacher evaluation results and research projects in progress at the university. On the
academic side, the information system allows for student access to medical records, course
materials, as well as online access to the university application. The information system as
implemented will increase transparency into outcomes, serving to create baseline data against
which the progress of future policy revisions can be considered.
Finally, the University recently began the implementation of a new, non-Mece2 funded, 5-year
humanities performance agreement designed with the Mece2 pilot model.
University of Tarapaca
Resources of $2.2M were allocated to the performance agreement with the University of the
Tarapaca with an additional $0.64M in funds provided by the university. These resources were
used to bolster the management capabilities of the university, to position the university as an
agent of academic integration with Peru and Bolivia, and to promote research in areas closely
linked to desert life: the indigenous Chinchorro culture, the properties of endogenous desert corn,
and the problems of arsenic and borum in the land and water.
To bolster the management capabilities of the university, a strategic and operational control
system was designed and configured by the third year of the performance agreement. The system
allows the university to better evaluate both the academic and general management within the
university through collection and analysis of data. The university community has previewed the
system, which is currently being implemented.
The university has successfully coordinated with institutions in Peru and Bolivia to increase
integration with them, and has used that integration process to bolster academic quality within
the university. The university‘s brand is now recognized by 65% of the twenty-six targeted
institutions in Peru and Bolivia, exceeding the goal of 30% acceptance. The indexed publications
from the university have increased by 43%, falling just short of the goal of a 44% increase, and
the count of faculty with doctorates has increased by 9%, just short of the target of 11%. Finally,
student mobility (exchanges and internships) between universities in Peru, Bolivia, and Chile has
increased significantly, with (approximately 9 times the baseline number of transfer credits).
30
Annex 3. Economic and Financial Analysis
The value of the Project has been assessed through two distinct NPV calculations, each focusing
on an outcome associated with Mece2. The approach is conceptually similar, if not the same, as
that originally used in the Project Appraisal Document (PAD). Estimates indicate Mece2 has an
NPV USD (2005) of $677 million. The funds associated with the Project had an NPV USD
(2005) of $80 million, suggesting that conservatively, the total leverage of Mece2 was
approximately 8.5. This conclusion is summarized below and explained in turn:
Mece2 has led to shorter-duration academic programs and quicker time-to-degree,
resulting in time savings and increased career earnings for graduates of tertiary education.
This benefit is estimated at NPV USD (2005) of $335 million.
Mece2 has increased tertiary education quality, leading to an increased premium for wage
earners with tertiary education. Thirty percent of this benefit was assumed to be directly
attributable to the Project and its value is estimated at NPV USD (2005) $342 million.
As per the PAD, Mece2 is also expected to result in additional value by increasing
graduation rates; this effect (and its value) cannot yet be observed. Because Mece2
targeted its efforts to increase retention rates on first year tertiary education students,
beneficiaries have not yet graduated.
Increased Earnings from Shortened Education Duration
A significant focus area of the project was shortening the duration of tertiary education in Chile.
Historically, university study required an investment of approximately seven years. By reducing
time-to-completion, students spend less money and less time in acquiring their education.
Earnings increase because the avoided study time can be productively allocated to work. The
NPV of the project was calculated in part by examining the value of these incremental earnings.
Data collected through the SIES (funded by subcomponent 1.4) was used to complete this
analysis. Specifically, an analysis of time-to-degree across a fixed set of study-areas was
conducted for 2007 and 2009.4 The result was a shortening of 0.4 semesters (2.5 months) in the
weighted average of program durations, which means increased earnings for 2008-2017
graduates of US$692 million. 5 Though general funding for tertiary education increased in Chile
over the Mece2 project, no funds targeted this particular goal, suggesting that the most likely
explanation for the majority of these increased earnings is the impact of the Project. Figure 2
illustrates the shortening of degrees between 2007 and 2009.
4 Study areas account for most high enrollment degree programs, including: Architecture, Accounting,
Law, Emergency Medicine, Commercial Engineering, Construction Engineering, Journalist, Psychologist,
and Social Worker, as well as 43 others. 5 The number of 2009 graduates by program was used to create a weighted average duration of a
university education. If 2007 graduation numbers had been used in addition to those from 2009, student
program selection bias across years may have altered the outcome.
31
Figure 2. Weighted Average of Education Duration for a Sample Set of Study Areas
Figure 3. Program Length in Semesters, 2007 vs. 2009
As can be seen from figure 3, the reduction in program length was not evenly distributed among
all programs. The line follows y=x; points above the line represent programs that have
lengthened marginally (11), and points below the line indicate programs that have shortened (41).
Because the change in program duration was not consistent, a weighted average of the reduction
in program duration by number of 2009 graduates in each program was used. This was done in
order for the weighted average to reflect the true shortening of the programs as experience by
graduates in 2009.
The reduction in study length between 2007 and 2009 allows students to work and earn for an
incremental 2.5 months on average. An average monthly salary and an employment rate were
13
13.1
13.2
13.3
13.4
13.5
13.6
13.7
2007 2009
Sem
este
rs
Year of Graduation
y=x
Electrical
Engineer
Acting
Law
32
calculated for 2009 university graduates, again from the SIES data. The salary for the 2.5
incremental months of work was crossed with actual (as available) and projected university
graduates from 2008 until 2017, and discounted to 2005 values, as was done for the NPV
calculation in the PAD. Though benefits would continue to accrue to graduates after 2017,
uncertainty around projections would increase significantly. Restricting the outlook to a single
decade span means the estimate of program NPV is more conservative than it might otherwise be.
A number of assumptions were made in calculating the NPV associated with this aspect of the
project:
The size of each year‘s graduate cohort is based on the average growth rate in CRUCH
graduates from 2002 to 2009, which was 5.2% (slightly higher than the projected rate of
4.6% used in the PAD).
The employment rate was assumed to be 81.8 percent, which was the weighted average
employment of graduates from all university programs in 2009, and was marginally
lower than the assumption from the PAD.
The weighted average salary earned by 2009 graduates was based on SIES data, and was
determined to be Ch$622,694.
First year wages were assumed to grow at an annual rate of 2.5%, (the per capita growth
of the economy), as assumed by the PAD.
The discount rate is 6%, as in the PAD, and the exchange rate is taken as Ch$ 550,
marginally lower than that of the PAD.
Table 2 shows these assumptions result in NPV US $2005 of $669 million in increased earnings
for 2008-2017 graduates. Not more than half of this increase, or $335 million, is likely
attributable to the project.
Table 2. Earnings from Decreased Education Duration
Year CRUCH
Graduates
NPV
(USD 2005 million)
2008 32,024 61.4
2009 33,966 63.0
2010* 35,732 64.1
2011* 37,590 65.2
2012* 39,545 66.3
2013* 41,601 67.5
2014* 43,765 68.6
2015* 46,040 69.8
2016* 48,434 71.0
2017* 50,953 72.3
Total 669.2
Percentage attributable to Mece2 50%
Amount attributable to Mece2 334.6 Note: considers graduates from Consejo de Rectores universities, assumes new graduates growing at 5.2% (average
2002 to 2009), initial mean wage of M$1,132, employment rate of 81.8%, US$ 1.0 = Ch$550. * = projected
33
The NPV of this calculation compares favorably with the one estimated in the PAD; the realized
NPV of 669.2 (millions $2005) is well in line with the PAD estimate of 615 (millions $2005) for
a reduction in degree duration of 2.5 months. Exchange rate effects slightly raised the realized
NPV, but not by more than 10%. The disbursed funds of Mece2 had an NPV of 80 (millions
$2005); if half of the shortening of program length is attributable to the project, leverage of 4.2X
from the funds has been realized. In general, tertiary education funding has increased
significantly since program implementation, as a new student loan program has resulted in
significant increases in enrollment as well as increases in funds available to universities to
bolster quality. However, given the focused nature of Mece2 and the more diffuse nature of those
other funds, attributing half of the present value of the improvement to Mece2 appears to be
conservative.
The granularity of the data was complete only to the university level. Analyzing just the subset
of CRUCH universities instead of all the universities (Cruch & private) would have been
preferable, however, as most of the funds associated with Mece2 accrued to CRUCH universities.
This may make the NPV more conservative, as CRUCH degree durations would be expected to
have decreased more than the university-wide average.
Increased earnings from Higher Tertiary Wage Premium
A significant focus area of Mece2 was increasing the quality of tertiary education. Improved
quality is expected to increase the relevance of tertiary education for labor market outcomes. As
the quality of tertiary education increases, the wage premium for having completed tertiary
education is expected to rise, with employers willing to pay increasing premiums for graduates
with well developed and highly relevant skills. The NPV of the project was calculated in part by
examining the value of these incremental earnings.
Data collected through the SIES was used to complete this analysis. Since Mece2‘s funds went
almost entirely to universities (>90%), the wage premium driven by this program was calculated
by comparing earnings of university graduates to earnings of graduates from technical training
centers and professional schools. Specifically, the analysis used information from students who
had graduated between 2005 and 2008 and compared the growth in salaries earned in the first
year after graduation. The compensation data indicated that technical and professional school
graduates saw their incomes appreciate by 4% from 2005 to 2008. University graduates enjoyed
an increase in compensation of 5.5%, or an incremental salary growth of 1.5 percentage points
by 2009.
The value of the incremental 1.5 percentage points in wage premium from 2005 translates into
1.4% of the nominal 2009 salary. A number of assumptions were made to assess the impact of
this premium on the earnings of university graduates over the decade from 2008 to 2017.
Because the graduates are expected to benefit from the wage premium over their entire career
[30 years], the assumptions calculate the present value of the wage premium in the year of
graduation; future year premiums are discounted. The rest of the assumptions are detailed below.
The same assumptions used for calculating increased wages from shortened study-spans are used
here, as well as a few supplementary assumptions:
34
In addition to the 2.5% annual growth rate applied to first year wages, through the 30
year employment span, a 1.5% annual lifecycle earnings increase was projected.
As in the PAD, a discount rate of 6% is applied. This rate is used to normalize present
value earnings from 2008-2017 back to USD 2005 and to calculate the 30-year present
value of the wage increase for each graduation cohort.
Table 3 shows these assumptions result in NPV US $2005 of $1,026 million in increased
earnings for 2008-2017 graduates. Not more than a third of this increase, or $342 million, is
likely attributable to the project.
Table 3. Increased Wage Premiums Associated with Tertiary Education Quality
Year CRUCH
Graduates
NPV
(USD 2005 million)
2008 32,024 94.2
2009 33,966 96.6
2010* 35,732 98.2
2011* 37,590 100.0
2012* 39,545 101.7
2013* 41,601 103.4
2014* 43,765 105.2
2015* 46,040 107.0
2016* 48,434 108.9
2017* 50,953 110.7
Total 1,025.9
Percentage attributable to Mece2 33%
Amount attributable to Mece2 342
Note: considers graduates from Consejo de Rectores universities, assumes new graduates growing at 5.2% (average
2002 to 2009), initial mean wage of M$1,132, employment rate of 81.8%, US$ 1.0 = Ch$550, assumes 2.5%
continued initial salary growth, with incremental 1.5% lifecycle growth. * = projected
This estimate of $1,025 million 2005 USD compares favorably with the PAD estimate of an
NPV of ~$828 million 2005 USD (extrapolated for 1.4% growth from 1.5% and 0.5% growth
calculations). Some input assumptions changed slightly; undergraduate enrollment and
graduation have increased more rapidly than projected, and salary and exchange rates have
varied. Each of these factors appears to account for half of the difference between the initial
project and the current calculation.
As was the case with the value of the shortened education duration, other factors are likely to
have contributed to the realization of the project goals in increasing wage premiums and
educational quality. Endogenous factors include the student loan program cited above, which has
dramatically increased university enrollment and funds available to universities. Additionally,
exogenous factors, including Chile‘s shift to a knowledge economy, are also expected to increase
the premium associated with tertiary education. For these reasons, attributing a third of the
benefit to Mece2 appears to be a conservative approach to assessing the total value of the project.
35
Increased Earnings from Higher Graduation Rates
A significant focus area of Mece2 was increasing resources available for remedial tertiary
education and other programs intended to decrease tertiary drop out. Historical graduation rates
in Chile have varied between fifty and seventy-five percent, though reliable data in this area is
limited. Graduates of tertiary education enjoy a significant wage premium over those with only
partial completion of tertiary education. Increasing the graduation rate would thus have the effect
of moving more wage-earners into a higher income group. These incremental wages would
translate into a significant source of NPV for the project.
The long duration of Chilean tertiary education is a limiting factor in calculating the realized
value of Mece2. With programs commonly lasting seven or more years, students who
matriculated in the year Mece2 began will not graduate until 2013 at the earliest. If the effects of
Mece2 were delayed to any extent by the disbursement schedule, changes to graduation rates
may well not be observed until 2014-2015. As more information accrues, it will be possible to
further assess the contribution of a lowered attrition rate to the NPV of the Mece2 project, but at
present, any increase in graduation resulting from the project cannot yet be observed. Per the
calculations in the PAD, as the benefits of a higher salary are enjoyed by a wage earner over
their entire lifetime, and the premiums for complete vs. incomplete tertiary education are
significant, even very marginal changes to graduation rates would lead to realized NPV of
several multiples of the Mece2 investment.
36
Annex 4. Bank Lending and Implementation Support/Supervision Processes
(a) Task Team members
Names Title Unit Responsibility/
Specialty
Lending
Antonio Leonardo Blasco Sr Financial Management
Specialist LCSFM Fin. Management
Andreas Blom Sr Education Econ. SASED Education Econ.
Max Brennan Consultant LCSHE Consultant
Maria E. Castro-Munoz Sr Social Scientist LCSSO Social Scientist
Maria Lucy Giraldo Senior Procurement Specialist LCSPT Senior Procurement
Specialist
Lauritz B. Holm-Nielsen Consultant LCSHE Consultant
Jette Samuel Jeppesen Consultant LCSHE Consultant
Marta Elena Molares-Halberg Lead Counsel LEGES Lead Counsel
Kristian Thorn Consultant ECSHD Consultant
Morag N. Van Praag Senior Finance Officer CTRDM Finance
Martha P. Vargas Program Assistant LCSDE Assistant
Supervision/ICR
Diego Ambasz Operations Officer LCSHE Operations
Antonio Leonardo Blasco Sr Financial Management
Specialist LCSFM
Financial
Management
Alejandro Caballero Sr Education Spec. LCSHE Education Spec.
Maria E. Colchao Senior Program Assistant LCSHE Assistant
Maria Lucy Giraldo Senior Procurement Specialist LCSPT Procurement
Ana Maria Grofsmacht Procurement Analyst LCSPT Procurement
Jesko S. Hentschel Sector Manager ECSH4 Sector Manager
Daniela Marotta Country Economist MNSPR Country Economist
Omar Arias Sector Leader LCSHD Sector Leader
Chingboon Lee Sector Manager LCSHE Sector Manager
Janet Entwistle Senior Operation Officer LCSHE Operations
Livia Benavides Lead Operations Officer LCC6C Operations
Luz Maria Meyer Junior Professional Associate LCSFM Junior Professional
Associate
Michael Crawford Sr. Education Specialist LCSHE Team Leader
Maria Paulina Mogollon E T Consultant LCSHE Consultant
Marta Elena Molares- Halberg Lead Counsel LEGES Lead Counsel
Luis M. Schwarz Senior Finance Officer CTRFC Finance
Alejandro Roger Solanot Sr Financial Management LCSFM Financial
37
Specialist Management
Kristian Thorn Consultant ECSHD Consultant
Morag N. Van Praag Senior Finance Officer CTRDM Finance
Martha P. Vargas Program Assistant LCSDE Assistant
Lourdes Linares Sr. Financial Management
Specialist LCSFM
Financial
Management
Mariana Montiel Sr. Counsel LEGLA Sr. Counsel
Cidalia Brocca Finance Analyst CTRDM Disbursement
Patricia de la Fuente Hoyes Sr. Finance Officer CTRFC Disbursements
Christopher Sharp Consultant LCSHE Consultant
Maria Elena Paz Gutzalenko Program Assistant LCSHE Assistant
(b) Staff Time and Cost
Stage of Project Cycle
Staff Time and Cost (Bank Budget Only)
No. of staff weeks USD Thousands (including
travel and consultant costs)
Lending
FY05 25 145
FY06 4 30
Total: 29 175
Supervision/ICR
FY06 16 50
FY07 15 90
FY08 9 58
FY09 33 176
FY10 44 169
FY11 39 149
Total: 156 692
38
Annex 5. Beneficiary Survey Results
(Non applicable)
39
Annex 6. Stakeholder Workshop Report and Results
(Non applicable)
40
Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR
IMPLEMENTATION COMPLETION AND RESULTS REPORT
LOAN (IBRD-73170) TO THE REPUBLIC OF CHILE: TERTIARY EDUCATION
FINANCE FOR RESULTS PROJECT- FIRST PHASE APL - MECESUP2 PROGRAM
1. Project Development Objective and Implementation
1.1 Project Development Objectives (PDO)
The program, known locally as Mece2 (Higher Education Improvement) Program, fulfilled its
original objective of supporting Chile's development into a knowledge-based economy by
strengthening its tertiary education system. To this end, the funding system for tertiary education
was revamped in order to: i) enhance the quality and relevance of tertiary education; ii) ensure
access to all talented individuals; iii) improve internal efficiency and strengthen accountability
for results and iv) support national innovation system by increasing the availability of advanced
human capital. Beyond the program, the goal was accomplished regarding the support of the
Chilean Government in completing the transition from the current incremental funding system
for quality tertiary education to a results and performance-based system applied to an agreed
proportion of base funding for universities.
1.2 Project Implementation
In general terms, Project implementation was highly satisfactory with no great setbacks. It
sustained the continuity of the main objectives set in the Mece2 Program (Loan 4404-CH) with
essentially the same organizational structure, projected its intentions more intensely towards
results-based management and public accountability and introduced Performance Agreements on
a trial basis.
Implementation was conducted with resources from Loan 7317-CH, as well as resources from
the Chilean Government.
At the Tertiary Education Institution (TEI) level, Institutional Coordination Units (ICU) were
used. Each ICU had an upstream role of helping TEIs prepare Performance Agreements and
sub-project proposals and a downstream role focused on implementation, fiduciary management
and M&E of sub-project and Performance Agreements activities.
In 2010 the new Government and the Administration of the Division of Higher Education
(Divesup) agreed to institutionalize the Mece2 Program in a Department of Institutional Funding
which manages all resources allocated to tertiary education (direct public support (AFD), indirect
public support (AFI), traditional funding for institutional development, Mece2 and other
ministerial funding), excluding student financial assistance. Thus, after 15 years of successful
operation, the Program has acquired the necessary organizational, administrative and financial
stability to sustain the quality enhancement of Chilean tertiary education.
The Project incorporated modifications to funding instruments: the Competitive Fund for
Projects pertaining to the Mece2 Program turned into an Academic Innovation Fund (FIAC),
now focused on strategic axes that mediate structural weaknesses in the tertiary education system,
41
highlighting results-based management and seeking the highest degree of replication and impact.
To this end, 5 competitions were held in the 2006-2008 period and USD 115 million was granted
to 371 TEI sub-projects. This award includes Technical Training Centres and both public and
private universities of Council of Rectors of Chilean Universities (CRUCH), as well as
accredited autonomous universities (in the area of pedagogical or Ph.D. programs). Additionally,
Performance Agreements were incorporated as a new instrument on a trial basis, which enabled
the successful exploration of negotiated funding means between the Ministry of Education and
the Universities to achieve results, performance or notable outcomes on a national and
institutional level.
With regard to quality assurance, the implementation of voluntary accreditation processes (based
on self-assessment and external verification) in the Chilean university system, fostered during
the Mece2 Program, eventually brought about Law No. 20.129 in 2006. This Law currently
directs the processes for institutions and undergraduate and postgraduate degree programs via the
National Commission for Accreditation (CNA).
Additionally, the new web portal www.mece2.com was implemented, incorporating modern
online database technologies. The aim is to ensure adequate public diffusion and transparency of
the information and results generated in the Project.
2. Performance Assessment
The Project accomplished its development objectives and successfully demonstrates an improved
means of supporting tertiary education in Chile. The foregoing may be corroborated by the
compliance with triggers which measured the progress towards the PDO: i) employers‘
perception of graduate quality according to a survey; ii) number of graduates from Technical
Training Centres and Professional Institutions transferred to universities; iii) socioeconomic
profile of first year tertiary education students; iv) first year drop-out rate at institutions; and v)
percentage of competitively awarded or performance-based tertiary education public funding.
2.1 Studies
Studies pertinent to decision-making and impact on the Project and on TEIs were conducted,
including the following: Determination of Causes of Student Drop-out Rate in First Year of
University (2008); University Leaders‘ Perception of the Division of Higher Education‘s Policy
Instruments (2008); Employer and Graduate Perception of Tertiary Education Graduate Quality
(2008); Review of Chilean Tertiary Education, commissioned to OECD and the World Bank
(2009); State of Development of Advanced Human Capital for Education, Research and
Innovation in Chile, commissioned to National Academy of Sciences USA (2008).
2.2. Outcomes and Impact.
The Project enabled the foundations to be laid for:
The development and strengthening of an incremental funding system based on
Performance Agreements;
The initial implementation on a national scale of a new curricular design for tertiary
education based on learning outcomes and demonstrable competencies in tune with
international standards;
42
The definition and initial implementation of an academic credit transfer system (SCT-
Chile, compatible with the European credit transfer system) which would allow national
and international student mobility and progress towards the articulation of the different
tertiary education levels;
The consolidation of improving tertiary education quality and its continuous innovation,
in accordance with TEI strategic planning and CNA regulations (CNA, Law no. 20.129).
All participating Institutions in the Mece2 Program strengthened their capacities as a result of the
actions implemented by the Project, particularly with the experimental Performance Agreements
described below.
2.2.1 World Bank-Ministry of Education.
Project implementation enabled the identification of weaknesses and shortcomings in both the
Ministry of Education and the World Bank, which in turn allowed the study of options to make
internal changes so as to guarantee greater success in FIAC and Performance Agreements sub-
projects. For instance, studies are being conducted regarding ways to accelerate administrative
management approval cycles, facilitate reprogramming, have support systems available (for
example, specialized area-specific databases), orientate towards outcomes (and not only towards
activities and processes) and expedite procurement and hiring procedures, which is being
planned for inclusion in the new loan to be negotiated in 2011 with the World Bank.
2.2.2 Ministry of Education-DIVESUP-MECE2.
Divesup, Ministry of Education and pertinent agencies‘ capacities have been strengthened to
develop policies, strategies and management skills which enable the accomplishment of agreed
PDO, namely:
Creation of a Research and Performance Agreement Unit which has provided Divesup
with support for Performance Agreement design, negotiation and monitoring, as well as
the preparation of terms of reference, tender and monitoring of studies , statistics
preparation, triggers and analysis of results and impacts generated in sub-projects from
FIAC, Performance Agreements and the Project.
Generation of information for decision-making in both Divesup and TEIs on the basis of
studies conducted and/or contracted by the Project to support specific areas pertaining to
tertiary education and to move towards a more ambitious organization, modernization
and allocation of resources.
Support of the quality assurance institutionality and accreditation established in Law No.
20.129, 2006 which currently directs institutional processes for undergraduate and
postgraduate degree programmes through National Commission for Accreditation.
Implementation of a Tertiary Education Observatory since August 2007 (today known as
the National Information Service for Tertiary Education (SIES)).
2.2.3 Tertiary Education Institutions (TEIs).
The Mece2 Program has had a huge impact on institutions. For instance:
43
Generation of capacities and enhancement of management of recipient TEIs. As a
prerequisite of applying to the Mece2 Program Competitive Fund, strategic institutional
plans and participation in program and institutional accreditation were required, as
developed in the Mece2 Program through FIAC and Performance Agreements, so as to
consider resources for the design and implementation of 18 Institutional Research Units
which support institutional decision-making based on evidence and 22 Teaching-
Learning Units to achieve teacher training and strengthen student learning and their
demonstrable competencies. These institutional research capacities did not exist before
the start of the Mece2 Program, implying new opportunities for duly informed decision-
making and policy making.
These new capacities represent potent leverage management mechanisms in the
university system. Their introduction has meant, to a larger or lesser extent, substantial
and fundamental changes in attitude and practices in many TEIs.
The existence of legal accreditation requests and mechanisms of tertiary education in
Chile has enabled the installation and formalisation of quality assurance at an institutional
and program level, which in turn increases the amount of information available to the
public regarding educational alternatives and a satisfactory international presence.
The generation of institutional auto-evaluation skills, including analysis, strategic
diagnosis and accreditation results.
Strengthening of academic innovation in recognising academic credits (SCT-Chile), of
national and international student mobility and consolidation, competency-based
curricular innovation for lifelong learning, support of academic management and
strengthening of technical and vocational training.
In universities with experimental Performance Agreements, management has been
improved since more information is now available. Thanks to Performance Agreements,
there is currently a greater capacity for analysis and evaluation of information which
allows information to be compared to international experiences, as well as to decision-
making regarding their implementation and monitoring.
It has been shown that Performance Agreements open up a space to strengthen leadership
within institutions, which is essential for their implementation. They thus fall into a
virtuous circle in which they act collectively and not as a group of independent units.
The application of Performance Agreements ―at the highest level of the organisation‖ in
TEIs, without the need for the creation of ad-hoc intermediary structures, has allowed
internal bureaucratic barriers to come down and instil a results-based management style
at management level, not as an inconsistent fact, but as an intrinsic part of proactive
management.
Performance Agreement implementation, complementing the FIAC competitive fund, is
allowing the Ministry of Education to discover resource allocation alternatives which
include intense components of results-based management and public accountability. This
44
learning shall be decisive when considering a future redesign of funding mechanisms of
Chilean tertiary education.
Performance Agreements represent an alternative for base funding, streamlining public
interest with university autonomy.
Finally, FIAC and Performance Agreement sub-projects have strengthened TEIs
undergraduate, graduate and management areas.
3. Project Management Assessment
3.1 General Coordination
A full-time, seasoned General Coordinator with knowledge pertinent to large-scale and high-
impact project implementation has been in charge under the direct leadership of the Head of
Divesup. The Head of Divesup, the General Coordinator and the Director of FIAC composed
the Project Coordination Committee. It is important to highlight that Dr. Ricardo Reich has
permanently been in charge of the Project has greatly supported its continued vision and
implementation. Finally, the institutionalization in 2010 of the Mece2 Program must be
highlighted. It became part of the Department of Institutional Funding of Divesup, also directed
by Dr. Reich.
The General Coordination, with the collaboration of the Program Administration Unit and the
Research and Performance Agreement Unit, has been able to keep abreast of the global
management of the Project and the World Bank fiduciary procedures, monitor implementation
and offer technical assistance to beneficiaries. The Project staff is renowned for their
consistency over time with no significant changes for over a decade (since its creation in 1999),
in both the General Coordination and the management of its subdivisions and work teams, and in
its procurement units and financial management units, maintaining ongoing close coordination of
the Project work and integration of the information. Today they all compose the Department of
Institutional Funding, dependent on Divesup.
3.2 Internal controls
Throughout the implementation of the Project, there were no changes in the internal control
policies of issues regarding sub-project and Performance Agreement monitoring, funding,
accounting or procurements, with only the necessary adjustments to adapt to World Bank
requirements, Controller General's Office and internal and Government administration. As for
Financial Management, they complied in both form and substance, and by the deadlines set,
which was acceptable to the World Bank on every occasion. Likewise, the Controller‘s audits
and World Bank missions did not report fiduciary breaches in the Loan.
3.3. Innovations in Monitoring and Evaluation (M&E)
As a means of strengthening M&E, certain innovations were highlighted: i) Creation and
maintenance of an online monitoring platform; ii) Evaluation and qualification of sub-projects;
and iii) Sub-project and Performance Agreement Mid-Term Evaluations have been essential for
determining implementation progress, the feasibility of completing initiatives within the agreed
terms and the implementation of application modifications.
45
3.4 Mid-Term Evaluations and Implementation Evaluation
Mid-Term Evaluations, project implementation and funding instrument (FIAC and Performance
Agreements) evaluation reports issued in collaboration with the World Bank, the International
Advisory Committee and external consultants, were always positive with greatly appreciated
recommendations and suggestions to enhance application.
3.5 Risky Projects and Adopted Measures
During Project application, no sub-projects or Performance Agreements suffered important risks;
only a few temporary resource transfer suspensions occurred as TEIs took necessary measures to
correct delays in progress and to face some management problems, which were all fully resolved
in each case. These measures were generally a result of strategic implementation after the Mid-
Term Evaluation.
3.6 Use of Loan
The loan from the World Bank and the contribution from the Chilean Government were fully
disbursed by the completion date (31/12/2010), when the proposed objectives and goals had been
met. Disbursements were made with no setbacks, given that there were no expenditure eligibility
problems in the Statement of Expenditure.
3.7 Audits
The Project fulfilled all audit requirements (intermediate or semesterly and final or annual), in
form, contents, substance and term which was acceptable to the World Bank in each case. It is
important to mention that the information required and figures calculated always coincided with
World Bank records and Controller audit observations were fully resolved.
3.8 Workshops and Seminars
A series of communicative activities and themed workshops were held in which experts, TEIs
and Ministry of Education authorities, and sub-project and Performance Agreement directors
took part so as to tackle relevant sector-based topics, achieve sub-project synergy in similar areas
and diffuse accomplishments, lessons learnt and best implementation practices; the aim was to
encourage their replicability and use in future debates on tertiary education policy. In total, 14
were held on Performance Agreements and 54 on FIAC.
3.9 Compliance with Safeguard Regulations
The Project incorporated a World Bank safeguard policy related to increasing tertiary education
quality in underrepresented regions, seeking ways of improving access to university education
for underprepared school leavers and non-university tertiary education students. To this end, a
Development Plan for Indigenous Peoples had to be drawn up with three main causes of action:
i) the design of analysis mechanisms to identify and promote indigenous students participation in
tertiary education; ii) incentives so that universities from chosen areas offer remedial learning
opportunities to indigenous students; and iii) studies on ways to reinforce grants for talented
individuals of indigenous origin.
With that in mind, the Project conducted the study ―Description and Evaluation of the
Educational Conditions and Access to Benefits of Indigenous Students in Chile‖, whose
objective was to make a statistical description of the socioeconomic and educational conditions,
46
and the access to benefits of secondary and tertiary education indigenous students in Chile. The
study is theoretically relevant not because it is recent, but also because indigenous people‘s
education conditions were vague to analysts: no clear and accurate data was available on the
subject. The census-like nature of the research, in addition to the comparatively good results of
the benefits policies in Chile, facilitates the improvement of the validity of the results and
contributes to the international debate on the effects of grants and benefits in the facilitation of
access to tertiary education. In practical terms, this study represents a substantial contribution to
public policy since it presents analytical elements regarding the impact of State-provided benefits
to compensate for the lack of access to tertiary education, which will enable them to be adapted,
thus increasing their efficiency and equity.
3.10 Lessons learnt in Performance Agreements
Performance Agreements are a valuable instrument of institutional change, which with sufficient
focus, greater participation from the Ministry of Education, sufficient adaptability and
simplification, may effectively contribute to bringing about a substantial improvement in tertiary
education in Chile. To this end, simplification of their focus and process towards a national
perspective is recommended.
3.10.1 The negotiators.
The negotiators in both the Performance Agreement design and implementation phase have
proven to be a key factor in the successful development of the new instrument. The Negotiation
Panel is also important. As part of the Panel, the General Coordination of Mece2 Program, in
addition to the Research Units and Performance Agreements, has played a fundamental role in
coordinating the design, monitoring and control efforts.
3.10.2 The negotiation.
For an effective negotiation, procedures must be systematically applied. The most important
appear to be: i) attaining effectiveness to establish agreements on ―notable outcomes‖ to be
reached with the Performance Agreement, ii) acquiring mutual Negotiator-TEI trust during the
negotiation, iii) having a monitoring system which enables enforcement of commitments and
programs, iv) integrating and organizing experiences of negotiators and Project technical team
for the different Performance Agreements. Likewise, Performance Agreement implementation
has enabled the conclusion that in a possible future round, strategies to simplify negotiation
processes and the Performance Agreements themselves must be sought, what and how any type
of support can be linked to which results and performance, the amount of institutional support
the government is prepared to provide must be defined and how much is needed to incentivize
TEIs to achieve mutually desired changes.
3.10.3 International Advisory Committee.
The universities with Performance Agreements have stated that the International Advisory
Committee‘s transparent visits, questions, observations, suggestions and reports have been useful
and have supported their leadership. According to participating universities, Committee member
peers were important counterparts who shared additional knowledge and experience.
47
3.10.4 Management Skills.
A successful Performance Agreements implies the parties involved committing to a change in
administrative structures. The limitations which compose and subsist in the system must be
resolved: University – Ministry of Education – World Bank.
3.10.5 Procurement procedures.
World Bank procurement procedures are not entirely appropriate for Performance Agreement
implementation given that they unnecessarily slow down the acquisition of goods and services.
Time is key to completion in a Performance Agreement. With regard to Public Market (the
national purchasing system of the Republic of Chile), adjustments must also be introduced to
adapt them appropriately to a Performance Agreement.
3.10.6 Information and communication.
The implementation of actions so that all Chilean universities have access to information of
Performance Agreement outcomes and limitations, in addition to their replicability in other
institutional contexts to support university management, data collection and analysis, research or
curricular redesign.
In a possible future round of Performance Agreements their current heterogeneous nature ought
to be maintained, although if possible, the operating mechanism should generally be simpler and
the Ministry of Education‘s focus via the Performance Agreement should be: i) university
management and management training; ii) fostering of research; iii) Supporting student
exchanges and mobility; and iv) long-term sustainability of these activities.
All these recommendations and learning experiences have been considered in the design of their
escalation and mentioned in the new Performance-based Agreement Program 2011-2014.
3.11 Lessons learnt in FIAC
The implementation of FIAC over more than a decade has resulted in several learning
experiences regarding the procurement of incremental resources by competitive mechanisms.
Below are some of the most important.
3.11.1 Definition of priorities
All FIAC competitions have been held with a clear definition of set by the Ministry of Education
and the Project. Thus, attention has been on structural problems in the Chilean tertiary education
system. The clarity of objectives, in addition to significant incentives, has provoked a positive
reaction from TEIs to administer change.
3.11.2 Stability and flexibility.
FIAC is a resource allocation instrument for academic innovation and quality improvement in
student learning, providing stability and flexibility. The fact that the main axes of intervention
have remained in place for several years has enabled a greater number of TEIs to tackle these
priorities more widely and deeply. On the other hand, flexibility has been decisive for
incorporating innovations and making adjustments to priorities in accordance with a reality
changing with progress, the experiences acquired and new challenges.
48
3.11.3 Selectivity.
Given that it is a competitive fund, the selection of the best sub-projects has been reflected in
greater efficiency in allocating resources (only the best sub-projects are supported), which also
implies selectivity in TEIs. They know that their sub-projects shall be selected based on their
quality; consequently, many have started to be concerned with selecting proposals that are worth
developing, writing and submitting to each Competition. Besides, since FIAC defines a
maximum threshold of resource requests to be presented (corresponding to 18% of the total
amount of resources to be allocated), TEIs of a certain size which have great capacities to
prepare sub-projects, are forced to prioritize and select proposals internally and strategically
before presenting them to FIAC.
3.11.4 Evaluation of Impact.
The process to evaluate impact through specific sub-projects has begun in order to measure
impacts as a means of feedback and ongoing development of improvement systems.
3.11.5 Seal of rigorousness and transparency.
The rigorousness and transparency with which the competitions have been held have contributed
greatly to the credibility and seriousness of FIAC in the eyes of participants.
3.11.6 Implementation.
In the sub-project implementation process, a seal has been imprinted on the style of work. In
that way, all TEIs are aware of the rules and procedures employed for the submission of
proposals, their evaluation, selection and subsequent implementation. The various stages during
sub-project monitoring ensure that the rules are followed and known by all, and there is an
interest in achieving good results.
3.11.7 Principles of the Competitive Fund.
The total amount of lessons learnt shows that the success of an operation of this type lays in
rigorous compliance of the principles of a competitive fund; for instance, equal opportunities to
access funding resources, a transparent convocation process, evaluation and selection of
proposals, and the involvement on all levels of renowned professionals with impeccable
performance.
3.11.8 Equity and Quality.
Equity has been handled in different areas. Firstly, the 18% threshold enables smaller
institutions to allocate sub-projects. With regard to allocated areas, even though basic sciences
and engineering have stood out in their capacity of sub-project procurement, the support received
by the areas of education, arts, and humanities has been significant. Furthermore, it should be
noted that given the sub-projects implemented from Arica to Magallanes, there has been an
enhancement in the quality of training for students all over the country. On the other hand, the
concern, reviews, feedback and ongoing signs from FIAC and the project towards TEIs, with
regard to the quality of activities conducted in the sub-projects, have contributed to institutional
improvement about what results-based management and public accountability are.
Using thresholds for FIAC resource allocation has proven to be a powerful focalization
technique to specify a series of important public educational and social objectives related to
49
equity. The use of competitive funding instruments must carefully consider the public policies in
force in their design, in addition to the potential effect of individual threshold conditions (such as
relative development in regions, academic, scientific and technological production inventories)
on their expected results.
3.11.9 Administrative aspects.
Although it is considered that recipient TEIs do have professionals and a basic organisation that
supports sub-project directors in administrative aspects (through ICUs), sub-project
administrative responsibility is a challenge. The prerequisites and reviews of purchasing
processes by the World Bank, of local instances (Public Market) and within the same institutions,
in addition to a growing Controller audit, imply that great dedication is required in administrative
matters when conducting a sub-project. There are many procedures, accounting instances and
reports which must be prepared during the life of a sub-project. Most are necessary and
important. However, it should be possible to simplify processes and notify of requirements and
regulations beforehand so as to avoid an overemphasis on procedures and reports, and above all,
to focus on enhancing ICU institutional capacities. Good sub-project implementation and use of
allocated resources depend on their capacity to coordinate efforts and support any necessary
administrative processes. FIAC has noticed that when this institutional role is complied with,
better sub-project outcomes are achieved.
3.11.10 Networked Projects
TEI work has brought important benefits and challenges. Many networked sub-projects stand
out in both PhD and undergraduate programmes and also in the area of management. It has
strengthened links between colleagues at different institutions and has socialised and shared
experiences and ways to tackle common challenges. Also, it has allowed the achievement of
results and impacts that if they had not formed part of collaborative networked assignment, they
would not have been feasible. Some examples are the SCT-Chile system, the Qualifications
Framework and the institutional research networks, among others.
4. World Bank Performance Assessment
The Project deems its partnership with the World Bank as satisfactory. It particularly appreciates
the exceptional quality of its professionals and technical consultancy to the Project, as well as the
friendliness of their staff in their interpersonal relationships. Less favourable aspects include
important errors in the legal framework of the loan agreement, several changes of Project
manager and often inaccurate signs in procurement management according to World Bank
regulations, which in addition to structural faults in the Chilean fiduciary system have
unnecessarily complicated subproject and Performance Agreement implementation. The lessons
learnt in these areas are reflected in the fiduciary arrangements that have been pre-agreed with
the World Bank for the next phase of the project.
5. Arrangements for Future Implementation
By the termination date of Loan Agreement 7317-CH, the triggers and objective of the Project
were met, along with 100% disbursement on USD $25,130,000 loan and it used 100% of the
contribution of the Government of Chile USD $ 67,000,000. However, given that the sub-
50
projects are awarded by FIAC on a regular basis throughout the year and their term may also be
extended for reasons beyond TEI management, when the Loan expired, there were sub-projects
in progress to which the Chilean Government will contribute during 2011, 2012, and 2013 with a
sum additional to the one provided in the Loan Agreement about USD $ 32.436 billion to
complete it successfully and achieve its objectives, using the same sub-project implementation
strategies awarded with World Bank funds.
On the other hand, as there is concern regarding the sustainability of the 4 initiatives
implemented in the pilot Performance Agreement competition, which involved a commitment of
institutional resources beyond what was received to achieve and maintain effectiveness,
continuity actions will be implemented through total complementary contributions to the order of
USD $ 2 million, whose terms shall be set out in 2011 Competition Regulations with funding
from the Chilean Government.
Finally, as a way of sustaining the reforms initiated in the Mece2 program and strengthening
acquired institutional capacities, the new Government has decided to scale the pilot Performance
Agreement instrument with fiscal resources, while negotiations and the signing of a new loan
agreement with the World Bank for the 2012-2015 period is being finalized. To this end, in 2011
one or more Performance Agreement competitions will be convened in Teacher Training, Tuning
of Curricula, Internationalization of National PhD programs, Quality of Learning and
Management (FIAC2) and Technical and Vocational Training, which will be allocated in
accordance with the provisions of the Loan Agreement No. 7317-CH. As part of new
Performance Agreement orientation, it has been decided that FIAC will be updated to a second
version (FIAC2), introducing mechanisms for pre-selection of proposals, elements of negotiation
and a greater focus towards achieving outstanding results and performance.
51
Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders
(Not applicable)
52
Annex 9. List of Supporting Documents
1. Aide Memoire, Mid Term Review, January 2009
2. Avances de Implementación, Consejo de Rectores de las Universidades Chilenas,
Ministerio de Educación, Antofagasta, 2009
3. CAE, Chile‘s State guaranteed Student Loan Program Report, March 2011
4. CASEN-Survey 2009, 2006, 2003,and 2000
5. Convenios de Desempeño, Lecciones Aprendidas
6. CPS, Country Partnership Strategy Progress Report, 2010, 2011
7. Implementation Status Report or the Tertiary Education Finance for Results Project- First
Phase APL (P088498), January 2011, June 2010
8. Información del Comité Internacional MECESUP2, sobre convenios de desempeño y
triggers, July 2010
9. Informe del Comité Internacional MECESUP 2 (Enero 2009)
10. Mid-Term Review report for the Tertiary Education Finance for Results Project- First
Phase APL (P088498), June 25, 2009.
11. Programa Mecesup2. BIRF 7317-CH, Fondo de Innovación Académica, Lecciones
Aprendidas
12. Project Appraisal Document for the Tertiary Education Finance for Results Project- First
Phase APL (P088498), May 19, 2005.
13. Reich Albertz, Ricardo, Convenios de Desempeño 2011-2014, Formación de Profesores,
April 2011
14. Reich Albertz, Ricardo, ―Bases y Desafíos de la Aplicación de Convenios de Desempeño
en la Educación Superior de Chile, Documento de Posición‖
15. www.futurolaboral.cl/
16. www.sies.cl
Atta
cam
a D
esert
An
de
s M
ou
nt a
i ns
An
de
s
Mo
un
t ai n
s
Nevado OjosNevado Ojosdel Saladodel Salado(6,880 m)(6,880 m)VallenarVallenar
OvalleOvalle
ChillánChillán
VictoriaVictoria
OsornoOsorno
PuertoPuertoQuelliónQuellión
Puerto AisénPuerto Aisén
ManatialesManatiales
PuertoPuertoYungayYungay
CalamaCalama
SANTIAGOSANTIAGO
XIIXII
XIXI
X
IXIX
XIVXIV
VIIIVIII
VIIVII
VIVIRMRM
V
IVIV
IIIIII
IIII
I
XVXV
Vallenar
Ovalle
Chillán
Victoria
Osorno
PuertoQuellión
Puerto Aisén
Manatiales
PuertoYungay
Calama
Tocopilla
Iquique
Antofagasta
Copiapo
La Serena
Valparaíso
Concepcíon
Temuco
Puerto Montt
Valvidia
Coihaique
PuntaArenas
Talca
Rancagua
Arica
SANTIAGO
XII
XI
X
IX
XIV
VIII
VII
VIRM
V
IV
III
II
I
XV
ARGENTINA
BOLIVIA
PARAGUAY
PERU
URU
GUAY
Cape Horn
Strait ofMagellan
PACIFIC OCEAN
ATLANTICOCEAN
To San Sebastián
To Río Gallegos
To Paso
Río Mayo
To Nahuel Huapí
To BardasBlancas
To Villa Nueva
To Rodeo
To Fiambalá
To Salta
To Uyuni
To Corque
To La PazTo
Tacna
To Tolar
Grande
To Las Lajas
Atta
cam
a D
esert
An
de
s M
ou
nt a
i ns
An
de
s
Mo
un
t ai n
s
Nevado Ojosdel Salado(6,880 m)
85°W 80°W 75°W 70°W 65°W 60°W
40°S
35°S
30°S
25°S
50°S
45°S
40°S
35°S
30°S
25°S
20°S
80°W 75°W 70°W 65°W 60°W 55°W 50°W
CHILE
TARAPACÁ
ANTOFAGASTA
ATACAMA
COQUIMBO
VALPARAÍSO
REGIÓN METROPOLITANADE SANTIAGO
LIBERTADOR GENERALBERNADO O’HIGGINS
MAULE
BIO BÍO
LA ARAUCANÍA
LOS LAGOS
AISÉN DEL GENERAL CARLOSIBÁÑEZ DEL CAMPO
MAGALLANES Y DE LAANTÁRTICA CHILENA
LOS RÌOS*
ARICA-PARINACOTA*
I.
II.
III.
IV.
V.
RM.
VI.
VII.
VIII.
IX.
X.
XI.
XII.
XIV.
XV.
REGIONS OF CHILE
* Regions approved by cabinet and parliament, will become effective on October, 2007.
This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other informationshown on this map do not imply, on the part of The World BankGroup, any judgment on the legal status of any territory, or anyendorsement or acceptance of such boundaries.
0 200 400
0 200 400 Miles
600 Kilometers
IBRD 33386R
JUNE 2007
SELECTED CITIES AND TOWNS
REGION CAPITALS
NATIONAL CAPITAL
RIVERS
MAIN ROADS
RAILROADS
REGION BOUNDARIES
INTERNATIONAL BOUNDARIES
FALKLAND ISLANDS (ISLAS MALVINAS)A DISPUTE CONCERNING SOVEREIGNTY OVER THE
ISLANDS EXISTS BETWEEN ARGENTINA WHICH CLAIMSTHIS SOVEREIGNTY AND THE U.K. WHICH ADMINISTERS
THE ISLANDS.
CHILE