Completion Report

Project Number: 31081 Loan Number: 1792 September 2008

Indonesia: Technological and Professional Skills Development Sector Project

CURRENCY EQUIVALENTS

Currency Unit – Indonesian Rupiah (Rp)

At Appraisal At Project Completion 31 Dec 2000 31 Dec 2007

Rp1.00 = $0.000106 $0.000106 $1.00 = Rp9,455 Rp9,405

ABBREVIATIONS

ADB - Asian Development Bank BAN - Badan Akreditasi Nasional (National Accreditation Board) BAPPENAS - Badan Perencanaan Pembangunan Nasional (National

Development Planning Agency) BHE - Board of Higher Education CPMU - central project management unit DGHE - Directorate General of Higher Education EA - executing agency GDP - gross domestic product HELTS - Higher Education Long-term Strategy LTDPHE - Long-term Development Plan for Higher Education MIS - management information system MOF - Ministry of Finance MONE - Ministry of National Education MOU - memorandum of understanding M&E - monitoring and evaluation PAC - project advisory committee PCR - project completion report RRP - Report and Recommendation of the President to the Board of

Directors SPMU - sub-project management unit

NOTE

In this report, "$" refers to US dollars.

Vice President C. Lawrence Greenwood, Jr., Operation Group 2 Director General A. Thapan, Southeast Asia Department Director J.A. Nugent, Indonesia Resident Mission Team leader W. Kubitzki, Portfolio Management Specialist, Indonesia Resident Mission Team members D. Prijomustiko, Project Officer, Indonesia Resident Mission

CONTENTS

Page

BASIC DATA ii

MAP vi

I. PROJECT DESCRIPTION 1 II. EVALUATION OF DESIGN AND IMPLEMENTATION 2

A. Relevance of Design and Formulation 2 B. Project Outputs 4 C. Project Costs 9 D. Disbursements 10 E. Project Schedule 10 F. Implementation Arrangements 10 G. Conditions and Covenants 11 H. Consultant Recruitment 11 I. Performance of Consultants, Contractors, and Suppliers 12 J. Performance of the Borrower and the Executing Agency 12 K. Performance of the Asian Development Bank 13

III. EVALUATION OF PERFORMANCE 13 A. Relevance 13 B. Efficiency in Achieving Outcome and Outputs 13 C. Preliminary Assessment of Sustainability 15 D. Impact 16

IV. OVERALL ASSESSMENT AND RECOMMENDATIONS 16 A. Overall Assessment 16 B. Lessons 17 C. Recommendations 17

APPENDIXES 1. List of Higher Education Institutions 19 2. Output Target and Achievement 22 3. Number of Proposals and Awarded Grants 26 4. Summary of Retooling Programs 27 5. Summary of Staff Development 28 6. Allocation of Staff Development by Higher Education Institutions 29 7. Study Program Accreditations 2008 31 8. Summary of Loan Utilization 35 9. Summary of Disbursements 36 10. Project Implementation Schedule 37 11. Status of Compliance with Loan Covenants 38 12. Consulting Services 46

BASIC DATA A. Loan Identification 1. Country 2. Loan Number 3. Project Title 4. Borrower 5. Executing Agency 6. Amount of Loan 7. Project Completion Report Number

Republic of Indonesia Loan 1792-INO Technological and Professional Skills Development Sector Project Government of Indonesia Directorate General of Higher Education-Ministry of National Education $180 million PCR: INO 1054

B. Loan Data 1. Appraisal – Date Started – Date Completed 2. Loan Negotiations – Date Started – Date Completed 3. Date of Board Approval 4. Date of Loan Agreement 5. Date of Loan Effectiveness – In Loan Agreement – Actual 6. Closing Date – In Loan Agreement – Actual – Number of Extensions 7. Terms of Loan – Interest Rate – Maturity – Grace Period

26 July 2000 18 August 2000 23 October 2000 24 October 2000 29 November 2000 2 March 2001 4 June 2001 4 June 2001 30 June 2007 31 December 2007 1 Pool-based variable lending 25 years 6 years

9. Disbursements a. Dates Initial Disbursement

21 August 2001

Final Disbursement

21 April 2008

Time Interval

80 months

Effective Date

4 June 2001

Original Closing Date

30 June 2007

Time Interval

72.8 months

b. Amount ($million)

iii

Category

Original

Allocation

Last Revised

Allocation

Amount

Cancelled

Net Amount

Available

Amount

Disbursed

Undisbursed

Balance CPMU 21.800 26.570 2.484 26.570 19.316 7.254SPMU 131.200 106.130 27.172 106.130 104.028 2.102Front End Fee 1.800 1.800 0.000 1.800 1.800 0.000IDC 25.200 25.200 2.004 25.200 23.196 2.004ASP – Civil Works 4.000 (1.385) 4.000 1.385 2.615ASP- Consultant 0.500 0.000 0.500 0.000 0.500ASP Scholarships 1.500 (0.972) 1.500 0.972 0.529 Total 180.000 165.700 29.303 165.700 150.697 15.004ASP = Aceh support program, CPMU = central project management unit, IDC = interest during construction, SPMU = sub-project management unit Source: Asian Development Bank estimates

10. Local Costs (Financed) - Amount ($) 35,405,389.00 - Percent of Local Costs 74.38 - Percent of Total Cost 23.49 C. Project Data

1. Project Cost ($000) Cost Appraisal Estimate Actual

Foreign Exchange Cost 137.000 115.292 Local Currency Cost 113.000 101.748 Total 250.000 217.040

2. Financing Plan ($million) Cost Appraisal Estimate Actual Implementation Costs Central Government Financed 50.000 54.784 ADB Financed 153.000 125.701 Participating Institutions Financing 20.000 11.559 Total 223.000 192.044 IDC Costs & Front end fee Borrower Financed 0.000 0.000 ADB Financed 27.000 24.996 Other External Financing 0.000 0.000 Total 27.000 24.996 ADB = Asian Development Bank, IDC = interest during construction Source: Asian Development Bank estimates 3. Cost Breakdown by Project Component ($000)

Component Appraisal Estimate Actual Central Project Management Unit Retooling Programs 4,320 13,073 Staff Development 1,678 0 Workshops 225 373

iv

Research and Studies 1,710 586 Consulting Services 3,193 1,308 Student Equity 18,000 11,544 Equipment and Furniture 437 309 Project Management 3,011 2,891 Sub-Project Management Unit In-Country Degree Fellowships 11,162 3,308 Overseas Degree Fellowships 23,325 27,823 In-Country Non-degree Training 6,690 7,972 Overseas Non-degree Training 3,000 3,891 Workshops 210 0 Researches and Program Development 3,496 12,669 Consultant Services 7,158 2,059 Civil Works 7,634 10,796 Instructional Materials 7,124 11,052 Equipment and Furniture 90,565 56,189 Project Management 21,088 7,005 Aceh Support Program Civil Works 0 1,067 Consulting Services 0 0 Scholarships 0 670 Front end fee 1,800 1,800 Interest during constructions 25,200 23,196 Taxes 8,911 17,459 Total 249,937 217,040 Source: Asian Development Bank estimates 4. Project Schedule

Item Appraisal Estimate Actual Project Management Consultants Date of Award Q2 - 2001 Q1 - 2005 Completion of Work Q2 - 2007 Q4 - 2007 Academic Consultants Date of Award Q1 - 2004 Q2 - 2005 Completion of Work Q4 - 2006 Q4 - 2006 Technical Audit Consultant Date of Award Q4 - 2002 Q1 - 2003 Completion of Work Q4 - 2006 Q1 - 2007 Retooling Programs Date of Award Q3 - 2001 Q1 - 2004 Completion of Work Q2 - 2007 Q3 - 2006 Student Equity Programs Date of Award Q3 - 2001 Q2 - 2003 Completion of Work Q2 - 2007 Q2 - 2007 Researches & Studies First Date of Award Q2 - 2001 Q3 - 2004 Completion of Work Q2 - 2002 Q3 - 2004 SPMU Batch 1 Date of Award Q3 - 2001 Q4 - 2001

v

Completion of Work Q2 - 2005 Q4 - 2005 SPMU Batch 2 Date of Award Q1 - 2002 Q4 - 2002 Completion of Work Q4 - 2005 Q4 - 2006 SPMU Batch 3 Date of Award Q1 - 2003 Q4 - 2003 Completion of Work Q4 - 2007 Q4 - 2007 Aceh Support Program - Date of Award - Q4 - 2005 Completion of Work - Q4 - 2007 Yogyakarta Support Program - Date of Award - Q1 - 2006 Completion of Work - Q4 - 2007 SPMU = sub-project management unit Source: Asian Development Bank estimates 5. Project Performance Report Ratings

Ratings Implementation Period

Development Objectives

Implementation Progress

From June 2001 to December 2007 Satisfactory Satisfactory D. Data on Asian Development Bank Missions

Name of Missiona

Date

No. of Persons

No. of Person-Days

Specialization of Membersb

Fact-Finding 12–28 April 2000 3 52 b and o Appraisal 26 Jul–18 Aug 2000 4 46 b, h, l, and m Inception 27 May–1Jun 2001 5 13 a, c, i, j, and k Review-1 28 Aug–15 Oct 2002 2 18 c and i Review-2 (Mid Term) Dec 2003–Jan 2004 3 52 d, i and m Review-3 8 Nov–10 Dec 2004 2 18 e and i Review-4 16 May–10 Jun 2005 2 19 e and i Review-5 3 Jan–28 Feb 2006 2 19 e and i Review-6 15 Jan–23 Feb 2007 2 15 g and i Review-7 16–29 Nov 2007 2 10 f and i Project Completion Review 3 73 f, i and n a fact-finding, appraisal, project inception, Mid-term review, project review mission and project completion review. b a – head portfolio management, b - senior education specialist, c – senior project implementation officer, d -

principal project specialist, e - government advisor, f - portfolio Management specialist, g – project implementation specialist, h - program officer, i - project officer, j – control and finance officer, k – control officer, l - financial analyst, m – higher education specialist, n – education specialist, o – staff consultant.

Source: Asian Development Bank

vi

I. PROJECT DESCRIPTION

1. At the project design stage in 1999, the economy in Indonesia was recovering from the financial crisis and forecasts projected average gross domestic product (GDP) growth of around 7% annually for the period 2000–2005. While the Government of Indonesia was prepared to improve the enabling conditions for private sector growth, a lack of adequately trained technicians, engineers, and entrepreneurial managers was a major concern for the business community and contributed to a growing mismatch between labor supply and demand. The country needed a labor force with the appropriate knowledge and skills to successfully exploit emerging economic opportunities in an increasingly globalized economy. The key issues of the higher education system at the project design stage were (i) weak linkages to the labor market, which resulted in study programs lacking in quality and relevance; (ii) poor learning and teaching environments in higher education institutions, including outdated teaching methods, and a lack of practical and hands-on training for students; (iii) obsolete equipment and under-utilized facilities; (iv) under-qualified lecturers and inefficient staff deployment; and (v) inefficient management of higher education institutions. 2. To stimulate sustainable economic growth and create employment opportunities, the Government launched a series of higher education reforms, including the Long-term Development Plan for Higher Education (LTDPHE) for 1996-2006.1 The major focus of the LTDPHE included (i) implementation of a new paradigm for good governance and improved management of higher education institutions, (ii) improvement of the relevance and quality of higher education study programs, and (iii) attainment of geographical and social equity within higher education. 3. To address the mismatch between labor supply and demand, the Government requested the Asian Development Bank (ADB) in 1998 to support strengthening the higher education system in accordance with the LTDPHE. The Technological and Professional Skills Development Sector Project aimed to improve Indonesia’s international competitiveness and contribute to sustainable economic growth and poverty reduction on the basis of gender, social, and geographical equity.2 4. The Project was approved on 29 November 2000 for $180 million from ADB’s ordinary capital resources and was declared effective on 4 June 2001. The original total project costs were estimated at $250 million, of which $180 million was financed from ADB loan proceeds, $50 million from the central Government budget, and $20 million from participating institutions. The executing agency (EA) of the Project was the Directorate General of Higher Education (DGHE). The Project consisted of two interrelated parts: Part A comprised strengthening the governance and management capacity of the public and private higher education systems, improving student equity, and supporting a retraining program; and Part B comprised strengthening selected higher education institutions (including the provision of educational equipment to bolster existing and develop new study programs), supporting staff development, upgrading study centers for women, and strengthening community and industrial relations. Under Part B, 74 higher education institutions received project support, of which 44 were public and 30 were private institutions. A list of higher education institutions receiving project assistance is in Appendix 1. 1 The LTDPHE focused on key indicators of achievement named as RAISE++ (R = relevance and quality;

A=Academic Atmosphere; I=Internal Efficiency; S=sustainability; E=Efficiency and productivity; plus access and equity, and leadership).

2 Report and Recommendation of the President to the Board of Directors for the Technological and Professional Skills Development Sector Project, November 2000.

2

5. In 2006, the Government requested a cancellation of anticipated loan savings of $14.3 million, reducing the loan amount to $165.7 million. To address urgent reconstruction and rehabilitation needs related to the 2004 tsunami in Aceh and the major earthquake in Yogyakarta in 2006, ADB approved the Government’s request for a change in scope to (i) assist two higher education institutions in Banda Aceh by reallocating a total of $6 million for civil works ($4.0 million), scholarship support ($1.5 million), and consulting services ($0.5 million); and (ii) support four higher education institutions severely affected by the Yogyakarta earthquake by reallocating $1.8 million to procure new equipment, rehabilitate buildings, and provide scholarships for students. The loan closing date was extended from 30 June 2007 to 31 December 2007 to complete ongoing civil works in Aceh and Yogyakarta. There was a final cancellation of $15 million at the project closing.

II. EVALUATION OF DESIGN AND IMPLEMENTATION

A. Relevance of Design and Formulation

6. The overall objective of the Project was to improve Indonesia’s international competitiveness and help achieve sustainable economic growth to contribute to poverty reduction that is equitable on a gender, social, and geographical basis. The purpose of the Project was defined as increasing the efficiency and effectiveness of higher education on a regional basis through improved quality, capacity, and relevance of the priority disciplines in the public and private higher education systems. 3 The Project objectives also included strengthening public–private partnerships in the higher education system.

7. The Project comprised two interrelated parts: Part A focused on strengthening the governance and management capacity of the public and private higher education systems at the national level, improving student equity, and supporting a retraining program; and Part B focused on strengthening 74 higher education institutions, which included the provision of training equipment to bolster existing and support new study programs in five priority disciplines, supporting staff development, upgrading study centers for women, and strengthening community and industrial relations. A summary of project outputs and achievements is presented in Appendix 2. 8. Overall the project design was relevant to improve the quality of the education system. The project’s objectives were consistent with the Government’s priorities and helped achieve the reform targets under the innovative LTDPHE. The LTDPHE, initiated in 1995, aimed at reforming the higher education system with a particular focus on improving the quality and relevance of higher education study programs, improving access to and the equity of higher education, and strengthening governance and accountability within higher education institutions. In the wake of the Asian financial crisis, the LTDPHE was revised in 2000 and 2003 in response to structural socio-economic and political changes.4 In addition, the project objectives were relevant and in line with the current Strategic Plan of the Ministry of Education 2005-20095, which focuses on (i) increasing access to and the equity of higher education, (ii) increasing higher education gross enrollment rates from 14.26% to 18.00%, (ii) reducing the percentage of unemployed higher education graduates by 50%, (iii) increasing the number of post-graduate lecturers from 50% to 70%, (iv) improving the capacity of private higher education institutions by

3 See RRP(this requires a proper citation, i.e. full title and date of RRP), Appendix 1: Project Framework. 4 The LTDPHE was later renamed the Higher Education Long-term Strategy (HELTS) (what year?) 5 Strategic Plan of the Ministry of National Education, 2005-2009.

3

20%, (v) increasing the number of study programs having A or B accreditation6 from 1,000 to 3,000, (vi) improving the literacy of students and lecturers on information and communication technology, and (vii) promoting at least four universities to become among the best 100 higher education institutions in Asia and/or the best 500 in the world. 9. In retrospect, the project design should have included an improved monitoring and evaluation (M&E) framework with a specific defined outcome, and outputs with indicators to facilitate assessments at the national and institutional level. Baseline indicators were determined at the higher education institutional level during the first year of an institution’s participation. While the M&E activities were carried out systematically at all participating higher education institutions, the M&E system was not adequately developed to assess final project impacts on (i) the higher education system at the national level, (ii) improvements to the country’s international competitiveness, and (iii) sustainable economic growth. Indicators were designed to collect quantitative information on the Project’s performance. However, the M&E system lacked indicators to assess qualitative improvements to the teaching and learning process, relevance of study programs, and increasing efficiency and effectiveness at the institutional level. 10. While the Project achieved success in improving teaching and learning environments at the 74 project institutions, the project completion report (PCR) Mission noted that the project design could have included a strategy to improve coordination mechanisms among higher education institutions and DGHE to harmonize curriculum revision, and developed study programs and strengthened instructional methods to raise quality standards in the higher education system country-wide. Due to a lack of suitable coordination mechanisms, achievements were largely limited to the 74 project institutions and accomplishments varied between participating higher education institutions. While the project objectives also included strengthening the Government’s policy regarding public–private partnerships in the higher education system, a clear strategy was not incorporated into the project design. Thus, the level and quality of interaction with the private sector to develop competency and skills standards, and regularly adjust study programs to reflect market requirements, varies among higher education institutions. 11. Many of the participating institutions experienced significant delays in procuring equipment and consultants. This was partly due to the absence of a detailed procurement plan in the project design and the lack of familiarity and skills on procurement management during implementation. The problems were rectified during a project review mission as reflected in the memorandum of understanding (MOU) of the review mission. 12. Selection of the higher education institutions to be supported by the Project was conducted through a competitive and transparent mechanism based on a set of published criteria and procedures as contained in the Report and Recommendation of the President to the Board of Directors (RRP), which remained valid during project implementation.7 All higher education institutions were invited to submit proposals to upgrade existing and develop new study programs, and develop institutional support systems. 8 The project received 1,865 proposals from higher education institutions. Following an assessment by independent academic reviewers, 170 proposals were approved for project assistance to strengthen current 6 Accredited A is excellent (score 401 to 500), and B is good (score 301 to 400) 7 Criteria for eligibility and evaluation of proposals are referred to in paragraphs 72 and 73, and further elaborated in

Appendix 7 of the RRP. Based on these criteria the EA prepared guidelines for sub-project preparation, and disseminated the guidelines to all participating institutions.

8 Institutional support systems include library, women study center, teaching-learning center, computer center, etc.

4

and develop new study programs, and 110 were approved to promote institutional support systems. These study programs and institutional support systems came from 74 higher education institutions, consisting of 44 public and 30 private institutions. To represent geographical equity, the higher education institutions were selected proportionally from the three regions of the country.9 Each higher education institution proposed detailed implementation and procurement plans, which were approved by the EA and ADB. The higher education institutions established sub-project management units (SPMU) and opened a bank account. Funds were transferred directly from ADB to SPMU accounts based on the agreed payment schedule. A summary of the number of proposals and selected higher education institutions is presented in Appendix 3. B. Project Outputs

13. Part A: Strengthening the Governance and Management Capacity of Higher Education Institutions, Improving Student Equity, and Supporting a Retraining Program. Part A of the Project was implemented by the central project management unit (CPMU) and included six areas: (i) accelerating policy reform in university governance and system restructuring, including financial management and evaluation capabilities, (ii) strengthening the CPMU’s management information system (MIS), (iii) strengthening the university accreditation system, (iv) supporting a student equity scheme, (v) retraining about 20,000 unemployed and underemployed graduates, and (vi) supporting two research studies.

14. Policy Reform in University Governance and System Restructuring. The Project was intended to support the Board of Higher Education (BHE) and Board of National Accreditation (BAN) in accelerating reforms in university governance and system restructuring. Staff of DGHE and members of BHE and BAN participated in international seminars and workshops related to governance and management of higher education systems in other countries. In addition, a national workshop on strategies to strengthen management of higher education institutions in Indonesia was held in March 2007. The Project contributed to increased knowledge among the relevant staff of DGHE, BHE, and BAN on matters related to management and governance in higher education. However, the direct impact of the Project on policy reform and system restructuring at the macro-level was very limited. As the LTDPHE was launched shortly before project commencement, the Project provided support on implementing strategies under the LTDPHE at the higher education institutional level, rather than developing policy reforms and restructuring the system.

15. Strengthening the MIS. The Project supported the establishment of an MIS at the CPMU and at all participating higher education institutions to enable the smooth flow of information to support decision-making and monitor project implementation. Hard- and software needed to set up the MISs were provided to participating institutions, and university staff were trained to administer an MIS. In addition, a web-based benefit monitoring and evaluation system housing project specific data was established at a majority of project institutions. The benefit monitoring and evaluation system provides quantitative information on study programs, enrollments, lecturers, equipment, and resource materials. However, it is mainly used at project institutions and not integrated into the overall MIS system of DGHE. Integration of the benefit monitoring and evaluation system is important to ensure that data and information are properly maintained beyond project completion.

9 Region I covered Sumatra, Central and West Kalimantan; Region II covered Java; while Region III covered Central

and East Kalimantan, Sulawesi, Bali, Nusa Tenggara, Maluku and Papua.

5

16. Accreditation System. The Project assisted DGHE to strengthen procedures to improve the rating system for the accreditation of study programs country-wide. While the previous system focused only on administrative aspects and quantitative information, the new system included a detailed procedure to evaluate the quality of study programs. To certify study programs, the Project supported the development of a strategy to establish an accreditation system certifying higher education institutions, which was approved in 2007.

17. Student Equity Scheme. At appraisal, the Project provided $18 million to finance scholarships for 20,000 students from low income families enrolled in higher education institutions. At project completion, $11.54 million in scholarships had been disbursed for 19,762 students, with about 42% were female and 58% were male students. Scholarships were awarded in the amount of Rp250,000 per month per student to support tuition and fees. The monthly scholarship rate was based on existing scholarships provided by DGHE and the Government was reluctant to increase the scholarship amount. As a result, the Project reached the targeted number of students with $11.54 million, saving $6.46 million of the original allocation of $18 million. The Government has confirmed that sufficient funds will be allocated in the future to enable the Ministry of National Education (MONE) to provide grants to qualified students from low income families after project completion.

18. While the student equity scheme supported students already enrolled in higher education institutions, the original Project design should have also included scholarships to help qualified senior secondary students enroll in higher education institutions. The Project subsequently developed a new scheme to promote qualified senior secondary graduates to enroll in higher education institutions and the Government has confirmed that the new scheme will be applied in the future.

19. Retooling Program. The Project originally provided $4.32 million to retrain about 20,000 unemployed and underemployed graduates in a broad range of skills. In 2005, ADB and the EA agreed to increase the allocation for retooling programs to about $13 million ($6.5 million financed by ADB and $6.5 million financed by the Government).

20. Retraining programs were conducted by public and private training providers in four batches to reach a total of 14,671 graduates. Participants of retooling programs were selected from among graduates of the priority study programs who had been unemployed or underemployed for the last 12 months. The duration of the retraining programs was 3 months. During the first two batches, the majority of the retraining programs were largely supply-driven and often based on allocated budgets and the available staff and equipment of the training provider rather than being market-driven and addressing the requirements of potential employers. Based on lessons learned and evaluation results, the Project improved the selection mechanism used to identify training providers, improved the curriculum and training methods, and strengthened links with local companies (e.g., internships and feedback from companies on improving the relevance of training programs). Overall, 4,317 participants of the retraining programs, or about 30%, found immediate employment. Appendix 4 provides detailed information on the retraining programs financed under the Project.

21. Research and Studies. The Project originally provided $1.71 million for two research studies on (i) relevant skill profiles for employability and (ii)tracer studies in higher education. At completion, the Project had funded six research studies in the following areas: (i) skill profiles and employability, (ii) public–private partnerships in education, (iii) competitive-based funding, (iv) quality assurance, (v) university autonomy, and (vi) benefit monitoring and evaluation. Even though the number of funded research studies had tripled, only $0.586 million or 35% of the

6

original allocation, was disbursed. Results of the studies were widely disseminated within the higher education system and actively discussed at several national seminars. The research outcomes were relevant and appreciated by decision makers at DGHE, and by managers throughout the higher education system.

22. Part B: Strengthening Existing Study Programs, Supporting New Study Programs, Upgrading Study Centers for Women, Supporting English Language, Upgrading Computer Laboratory and Information Technology Networks, and Strengthening Community and Industrial Relations and Teaching–Learning Centers. Part B was implemented through SPMUs established at every higher education institution supported by the Project. Based on a competitive selection mechanism, 74 higher education institutions were identified for assistance under the Project. Based on an approved procurement and activity plan (sub-project implementation plan), the Project transferred grants of $0.375 million to $5.750 million directly to the individual higher education institutions to finance (i) staff development; (ii) a strengthened process of curriculum review and revision, and course development; (iii) development of new instructional methods; (iv) upgraded facilities, including civil works and the procurement of equipment and furniture; (v) development of about 30 new study programs; (vi) assistance to selected study centers for women; (vii) establishment and/or strengthening of student advisory and job placement centers; (viii) development of computer laboratories and English language laboratories, management information systems, library networks and teaching–learning centers; and (ix) establishment and strengthening of community and business service centers.

23. Staff Development. The Project initially provided $44.18 million to upgrade the qualifications of 1,400 teaching staff, funding enrollment in 712 domestic and 286 overseas masters degree programs, 205 domestic and 99 overseas doctorate degree programs, and 98 technical–bachelor degree programs for teachers at polytechnic institutions. In addition, the Project provided in-country and overseas short-term training for lecturers and technicians. At completion, 1,226 lecturers had received scholarships for degree training, including 636 domestic masters, 229 overseas masters, 130 domestic doctorates, 211 overseas doctorates, and 20 technical–bachelor degrees. The project also provided technical short-term training (non-degree) to 779 teaching staff overseas and 3,229 teaching staff in-country. At completion, the Project had utilized a total of $43 million for teaching staff development. Appendix 5 provides a summary of staff development programs for overseas and domestic degree programs, and Appendix 6 provides a detailed overview of the distribution of staff development among higher education institutions. About 55% of overseas and domestic degree fellows have completed their studies and returned to their respective institutions. The Government has confirmed it will provide the necessary funding to ensure that participating fellows will be able to complete their studies in 2008. 24. The academic qualification, subject knowledge, and confidence of lecturers who received training under the Project have all improved. These lecturers have actively participated in improving the curriculum of their respective study programs. However, the impact of staff development programs on improving teaching methods varies. Feedback from review reports and field visits suggest that many lecturers still apply traditional teaching methods, devoting the most attention to theoretical knowledge. The management of participating higher education institutions needs to establish systems to promote modern teaching methodologies that focus on student-centered learning; encourage critical thinking, problem-solving, and team-building; and ensure that additional practical training is included in study programs. Some higher education institutions have successfully set up quality assurance systems to establish quality standards for teaching techniques, monitor the performance of lecturers in classrooms and

7

workshops, and provide assistance to all teaching staff to upgrade their teaching methods to meet accepted standards. Under the guidance of DGHE, information on these good practices should be disseminated within the entire higher education system. 25. Strengthening the Priority Study Programs, Supporting the Process of Curriculum Review and Revision, and Course Development. The Project supported 170 study programs in priority disciplines10 within 74 higher education institutions to strengthen their curriculum, improve links with industry and promote entrepreneurship development, and enhance capacity to manage workshops and laboratories. Total allocation for this sub-component was $3.6 million. In addition, the Project provided technical assistance (consultants) to the study programs, which was equivalent to $2.15 million. The Project successfully contributed to strengthening the quality and relevance of selected study programs as reflected by the improvement of the accreditation level of participating institutions from C to B (22 study programs), from B to A (20 study programs), and from non-accredited status to C, B, or A (34 study programs)11. An overview of the study programs developed and revised at each project institution, with respective ratings before and after the Project’s intervention, is in Appendix 8. 26. The PCR Mission noted that a considerable number of study programs for the same subject (e.g. civil engineering, electrical engineering, and mechanical engineering) were revised without coordinating efforts among each other and with DGHE. To ensure a uniform quality standard for study programs within the entire higher education system, it is important to establish a systematic approach for harmonizing curriculum development and revision, which should be closely monitored by DGHE to avoid the duplication of efforts. To regularly adjust study programs to changing requirements of the labor market, the increased involvement of industries and potential employers is also required. In addition, the integration between curriculum revision and strengthening teaching methodologies and instructional methods needs to be improved. The future curriculum revisions should integrate entrepreneurship development in all study programs to develop student understanding of the principles and practices of entrepreneurship. 27. Development of New Instructional Methods. The Project provided $2.6 million for teaching grants to higher education institutions. The funds were used by lecturers to improve teaching methods, including introducing new experiments and innovative exercises for students; developing new presentation materials and integrating information technology-based learning, group work, and student assignments. The Project organized 12 national workshops to exchange best teaching practices and innovative approaches. The teaching grants have contributed to improving instructional methods at individual project institutions and increasing the capacity of lecturers and students to undertake research and experiments, which is reflected by the growing number of publications in refereed journals as well as the amount of outside research grants received by participating institutions. Building on the successes and lessons learned in strengthening instructional methods, the following actions are recommended to develop a strategy to improve the teaching process throughout the higher education system to address issues related to uniform quality standards: (i) integrate curriculum development and teacher training, and (ii) promote active student-centered teaching and learning assessments. At present, achievements are limited to individual higher education institutions supported under the Project.

10 A list of priority disciplines supported under the Project is in Appendix 7. 11 Accreditation A is excellent (score 401 to 500), B is good (score 301 to 400), C is fair (score 201 to 300), and D is

unaccredited

8

28. Upgrading of Facilities, including Civil Works and Procurement of Equipment and Furniture. The Project provided $7.15 million for civil work and $56.19 million for equipment and furniture. In general, this component has proceeded according to plan. Civil works to rehabilitate existing facilities have been completed, and furniture and equipment have been procured and installed. However, the utilization of equipment and maintenance of workshops and laboratories remains a major concern. Monitoring reports and field visits have revealed that a considerable amount of new equipment at higher education institutions was not fully integrated into the teaching process, and consumables such as wood, metal, chemicals,etc. needed to utilize equipment meaningfully are often lacking. Lecturers need additional support to prepare teaching modules and lesson plans that integrate new equipment and enhance practical training for students. In addition, it was noted that workshops and laboratories at many higher education institutions are poorly maintained, operational manuals are lacking, and the cleanliness of workshops is an issue, all of which hamper the operation of equipment and inhibit a conducive learning environment. 29. Development of New Study Programs. The Project aimed to support the development of 30 new study programs in basic sciences, engineering, agriculture, law, management, and accounting. In total, 144 proposals for new study programs were submitted to the EA. Upon assessment by an independent academic review team appointed by the EA, only 12 new study programs were selected for support by the Project. This low success rate was primarily attributed to the weak institutional capacity of the applying higher education institutions and the anticipated low demand for graduates of the proposed new study programs. 30. Assistance to Selected Women Study Centers. The Project assisted six study centers for women through the minor rehabilitation of offices, procurement of equipment, and provision of small research grants. The centers conducted public awareness campaigns to promote greater enrollment of female students in priority study programs. It was reported that some centers undertook gender-related research and training, and become the regional references for local governments in relation to gender policy issues. 31. Creation and Strengthening of Student Advisory and Job Placement Centers. The Project supported student advisory and job placement centers at 12 higher education institutions through the provision of facilities and training of administrators. The centers have been well utilized by students and new graduates, and offer relevant advice and soft-skills trainings to improve their employability. Many centers cooperate with private companies to provide information on job vacancies and assist students to prepare for job interviews. However, the level of interaction with local industry and access to relevant labor market information need to be further strengthened. 32. Development of Computer Laboratories, English Language Laboratories, MIS, Library Networks and Teaching–Learning Centers. The Project supported the establishment of computer laboratories and MISs at 23 higher education institutions, set up 14 language laboratories, supported teaching and learning centers at 20 project institutions, and provided resource materials and software to 24 libraries. Computer laboratories are mostly in good condition and connected to the internet, and are fully utilized by students and lecturers. Libraries supported under the Project are mostly clean, air-conditioned, and fully utilized. However, in some libraries the registration of new books was seriously delayed and sample checks indicated low demand for newly procured resource books. Evaluation reports suggest that most of the teaching and learning centers at project institutions are not operating effectively and have only limited impact on improving applied teaching methods. A systematic approach is required to assist lecturers within the entire higher education system to apply modern, student-oriented

9

teaching techniques. DGHE is preparing to review the current in-service training system for lecturers. 33. Aceh and Yogyakarta Support Program. To respond to the Yogyakarta earthquake, the Project reallocated $1.8 million to help four universities: (i) State University of Yogyakarta; (ii) University of Muhammadiyah Yogyakarta; (iii) Universitas Islam Indonesia; and (iv) Universitas Pembangunan Nasional Veteran. The first three universities were included in the list of 74 higher education institutions supported under the Project. Major activities of the earthquake response included civil works for rehabilitation of buildings, purchase of equipment, and student equity schemes. All activities had been completed as of 31 December 2007.

34. To respond to the Aceh tsunami, the Project reallocated $6 million for civil works ($4.0 million), scholarship support ($1.5 million), and consulting service ($0.5 million). However, civil works for the new construction at Universitas Iskandar Muda and the rehabilitation of buildings at Universitas Syiah Kuala in Banda Aceh were significantly behind schedule. At the loan closing date of 31 December 2007, only about 60% of construction and rehabilitation works had been completed. The EA confirmed that completion of the reconstruction will be taken over by the respective universities using funds from the Government.

35. Good Governance. During implementation, the Project managed over 5,500 procurement contracts, mostly implemented by SPMUs at 74 higher education institutions throughout Indonesia. In 2005, ADB hired a staff consultant to conduct a procurement audit of a representative sample of contracts. The findings have been reported to the Integrity Division of the ADB’s Office of the Auditor General. From 2003 to 2007, ADB investigated 48 procurement cases, 23 of which were reported by Indonesia Resident Mission staff, 20 were investigated as follow-up from the procurement audit report, and 5 cases were reported by other parties. DGHE has also reported these cases to the Inspectorate General of MONE. To date, 35 cases have been closed with sanctions applied in 8 cases, and 13 cases remain to be resolved. To address issues related to procurement, the CPMU organized certified procurement training for project staff at all levels in May 2007.

C. Project Costs

36. At appraisal, the project cost—including tax, duties, and interest during construction—was estimated at $250 million equivalent, of which $137 million (including $27 million for interest during construction) was the foreign exchange component, and $113 million was in local currency. ADB provided $180 million (72% of the total project cost) from its ordinary capital resources to cover 100% of the foreign exchange costs and 38% of the local currency costs. The Government was expected to provide $50 million, while the participating higher education institutions were estimated to make contributions totaling $20 million. 37. In 2006, the Government requested to cancel anticipated loan savings. On 20 October 2006, the cancellation of $14.3 million was declared effective, reducing the loan amount to $165.7 million. At completion, the actual disbursement was $150.7 million (including interest during construction and the front-end fee), thus an additional $15 million was cancelled. In total, the Government contribution was $54.8 million in local currency, compared with the appraisal estimate of $50.0 million. The contribution of the higher education institutions is estimated at $11.6 million (60% of appraisal estimates). A detailed breakdown of projected and actual project costs is presented in Appendix 9.

10

D. Disbursements

38. Total disbursement of loan funds at completion was $150.7 million equivalent, or 83% of the loan amount at appraisal. Actual disbursement during the first two years was low, primarily due to the late recruitment of consultants and the slow process of procurement by higher education institutions. Disbursement rose steadily beginning in 2004, three years after project commencement, and reached a total of $150.7 million by 31 December 2007. There was no disbursement plan in the RRP, but the loan agreement indicated a target disbursement of $27 million, $81 million, $153 million, and $180 million by the end of years 1, 2, 3, and 4, respectively. Appendix 10 illustrates the actual disbursement as compared to its projection in the loan agreement. The loan proceeds for Part t B were disbursed through a direct payment mechanism from ADB to the accounts of higher education institutions. Under Part A, funds were disbursed through an imprest account established at Bank Indonesia, with the exception of consulting services, which were disbursed through direct payment. ADB provided an initial deposit in the amount of $1.5 million under the imprest account.

E. Project Schedule

39. The Project was approved on 29 November 2000. The loan agreement was signed on 2 March 2001 and became effective on 4 June 2001, six months after approval. The time lag between loan approval and loan signing of about 4 months was due to internal processes within the Government; an additional 3 months following the loan signing was required by the Government to meet conditions for loan effectiveness. The project loan was originally to be implemented over a period of 6 years lasting until June 2007. Due to the change in scope to support urgent reconstruction requirements at the disaster-affected institutions in Aceh and Yogyakarta, the loan closing was extended by six months to 31 December 2007. In general, all planned activities have been implemented. However, some activities under the CPMU and SPMU were implemented behind schedule, including strengthening MISs, revising the curriculum of selected study programs, and procuring new equipment. The actual and projected schedule of project implementation is presented in Appendix 11.

F. Implementation Arrangements

40. A CPMU was established to facilitate overall project implementation and manage project activities under Part A. The CPMU was headed by a full-time Project Director who was responsible directly to DGHE and MONE. Four professional staff from leading universities were hired to assist the Project Director in managing day-to-day project administration in the area of financial management, technical assistance, research and studies, and benefit monitoring and evaluation. In addition, a number of support staff were contracted to help administer the project. All staff engaged by the CPMU were financed under the central MONE budget.

41. A project advisory committee was established to provide policy guidance to the Project. The project advisory committee was chaired by the Director General of DGHE, and comprised members from DGHE, the National Development Planning Agency (BAPPENAS), Ministry of Finance (MOF), BAN, BHE, and other relevant professional organizations. During implementation, the chair of the project advisory committee closely monitored the project and provided policy direction as necessary, but the entire committee rarely met to discuss project-specific issues. In addition, quarterly monitoring meetings were held at BAPPENAS, where representatives from MOF and BAPPENAS received progress reports, and provided comments and guidance.

11

42. To implement activities under Part B, SPMUs were established at all 74 higher education institutions supported under the Project. The staff employed in the SPMUs were financed by their respective higher education institutions. In general, the CPMU and SPMUs were highly committed and managed project activities under their responsibilities well. Selection of the project higher education institutions was based on a competitive mechanism, which was earlier described in detail in paragraph 12.

G. Conditions and Covenants

43. The conditions for loan effectiveness were met by the Government within three months of loan signing, and loan effectiveness was declared effective without significant delay. The conditions for loan effectiveness included the (i) provision of a legal opinion by the Ministry of Justice, (ii) establishment of the project advisory committee, (iii) establishment of the CPMU, and (iv) selection of a technical experts panel. Prior to loan effectiveness, the EA had taken advance actions, including a socialization workshop on criteria and procedures for sub-project selection and proposal preparation.

44. In general, the loan covenants were complied with, as shown in detail in Appendix 12. Out of a total of 30 covenants, only two covenants were partially complied with. Schedule 5 paragraph 26 stipulates that the Government take measures to accelerate the implementation of university autonomy as mandated by Government Regulation No. 61/199912, and report to ADB on the progress of the actions. The Government promoted the autonomy of public higher education institutions throughout project implementation. However, there was no action plan submitted to ADB by the specified date of 31 March 2001. Schedule 5 paragraph 12 also stipulates that the Project will retrain 20,000 graduates. However, in 2005, ADB and the EA agreed to reduce the target to about 13,000 graduates. H. Consultant Recruitment

45. At appraisal, it was estimated that 884 person-months of national and 299 person-months of international consultants were required to support project management and academic services. Of these, 372 person-months of national and 60 person-months of international consultants were planned to be managed by the CPMU, while the remaining 472 person-months of national and 239 person-months of international consultants were allocated to the SPMUs. The consultants engaged centrally were intended to assist the CPMU in project implementation, quality assurance, monitoring and evaluation, and technical audit. Consultants at the SPMUs were primarily assigned to help strengthen the academic content of study programs at individual higher education institutions. The RRP did not specify the packaging and methods for the procurement of consultants. 46. During the Inception Mission in July 2001, ADB requested the EA to prepare a procurement plan for consulting services, and in 2002 the EA submitted a proposal for four packages: (i) Package-1: consultant team for independent audits; (ii) Package-2: consultant team to support project management services at the CPMU level; (iii) Package-3: academic consultants for engineering, basic sciences, and agriculture; and (iv) Package-4: academic consultants for social sciences. Bidding for Package-1 and Package-2 was conducted in September 2002. The contract for Package-1 (technical audit) was signed in January 2003 with Grant Thorntorn (Moores and Rowland). Recruitment of consultants under Package-2 (project management services) experienced significant delays, primarily due to an allegation of conflict

12 Government Regulation No. 61/1999 regarding University Autonomy

12

of interest in the selection process. ADB requested that the CPMU reevaluate the technical proposals for Package-2 and eventually approved the contract award to LAPI-ITB. As a result, the mobilization of the consultant team was delayed by about two years until February 2005. By then, most of the project activities that would have been supported by the consultant team were already ongoing and ADB advised the EA to adjust the terms of reference of Package-2 to reduce the consultant inputs. In 2002, it was agreed that the academic consultants under Package-3 and Package-4 would be engaged through a firm. However, due to significant delays in compiling consulting requests from all participating higher education institutions into two packages, the bidding process started two years behind schedule. In order to provide urgently needed consulting services to support higher education institutions in curriculum development, ADB supported the EA’s request to engage individual consultants in addition to the four packages. Consultants under Package-3 and Package-4 were mobilized in the second quarter of 2005. The recruitment of consultants has been implemented in accordance with ADB’s Guidelines on the Use of Consultants (2007, as amended from time to time).

47. Overall, the utilization of international consultants decreased from 299 person-months to 65 person-months, while the use of domestic consultants increased from 884 person-months to 1,228 person-months. This resulted in a reduction in the allocation for consulting services from $10.3 million to about $3.6 million. Appendix 11 provides an overview of the consulting packages, including planned and actual consultant inputs.

I. Performance of Consultants, Contractors, and Suppliers

48. The Project administered over 5,500 procurement contracts, which were mostly implemented by SPMUs at the 74 higher education institutions. In general, the performance of contractors to implement civil work and suppliers to provide equipment, furniture, and instructional materials was considered satisfactory. The contractor for the construction and rehabilitation of buildings at the University of Iskandar Muda and University of Syiah Kuala in Aceh failed to complete their respective tasks by December 2007 and consequently the CPMU imposed an appropriate penalty in accordance with the agreed-upon contract. The performance of consultants recruited under the four packages was partly satisfactory with respect to timeliness of delivery and quality of outputs. Due to delays in the recruitment process, some consultants proposed by the firms were no longer available for the project. The performance of replacements often did not meet expectations. J. Performance of the Borrower and the Executing Agency

49. The Borrower and the EA demonstrated a strong commitment to ensure successful implementation of the Project. Counterpart funds were provided in a timely manner and in an amount required by the Project. For some categories, such as the retraining program and project management, the amount of counterpart funds was higher than initial projections. In total, the Government provided $54.8 million equivalent of counterpart funds as compared to $50.0 million projected in the appraisal.

50. Representatives of the Borrower and EA met at least twice a year to discuss project progress, obstacles to implementation, and the allocation of an annual budget. The CPMU played an important role in implementing the Project with guidance and supervision from DGHE, BAPPENAS, and MOF. Overall, the performance of the Borrower and EA is rated satisfactory.

13

K. Performance of the Asian Development Bank

51. ADB provided strong guidance and support during project implementation through semi-annual review missions, and intensive consultations and meetings. Over the life of the Project, ADB fielded seven review missions including a mid-term review in January 2004. The mid-term review reported that many SPMUs were complaining of frequent delays in getting a response to their requests to the CPMU and ADB for contract approval and staff development clearance.13 The delays occurred primarily due to incomplete or inadequate bidding documents. Following a review mission in 2005, the CPMU and ADB agreed to improve their log and tracking system of incoming and outgoing letters to resolve the problem. Since then, about 90% of letters received by ADB were responded to within two weeks. ADB also demonstrated flexibility in responding to the Project’s requests with regard to the reallocation of loan proceeds among categories, including additional components for Aceh and Yogyakarta reconstruction efforts. Overall, the performance of the ADB is rated satisfactory.

III. EVALUATION OF PERFORMANCE

A. Relevance

52. The Project is rated as highly relevant. The Government considers reforming the higher education system as a key strategy to promote economic growth and improve international competitiveness, improve equitable access to income-generating opportunities to reduce poverty, and remove disparities between rural and urban regions to improve living conditions throughout the country. The project objectives were consistent with the Government’s priorities in the higher education sector. The Project supported the Government’s LTDPHE and the Strategic Education Plan of MONE 2005–2009, which includes reforming the higher education system with a particular focus on improving the quality and relevance of study programs, increasing access to and equity in higher education, and strengthening governance and accountability. 53. The Project strengthened the accreditation system, which is applied at public and private higher education institutions country-wide, to improve the quality and relevance of study programs, and significantly improved the teaching and learning environments in the 74 participating higher education institutions. Equitable access to higher education was improved by providing scholarships to 19,762 students from low income families, and by supporting less developed institutions across the three project regions. The Project has significantly contributed to increased access to and the improved quality of higher education, and the increased relevance of study programs. However, its contributions to improving Indonesia’s international competitiveness and sustainable growth cannot be quantified. B. Efficiency in Achieving Outcome and Outputs

54. In general, the Project was rated efficient in attaining its intended outcomes and outputs in a cost-effective manner. The Project was well managed by the CPMU, which was fully funded by the Government. At the institutional level, the Project was managed efficiently by SPMUs. ADB’s internal processing and administering of the project was considered effective and its responses to the EA’s inquiries were prompt and efficient.

13 Asian Development Bank’s record and Mid-term Review Report, January 2004.

14

55. Selection Mechanism. The competitive selection mechanism used to identify higher education institutions to participate in the Project proved to be successful in creating a strong sense of ownership and an entrepreneurial spirit at all 74 higher education institutions. Selection criteria were largely based on academic indicators of pre-selected study programs. In retrospect, more emphasis could have been placed on the overall management capacity of the respective higher education institutions and on labor market information in order to ensure that the study areas supported under the Project met the current and future demands of the industry. 56. Staff Development. Investments in staff development improved the academic qualification, subject knowledge, and confidence of lecturers who received training under the Project. Lecturers actively participated in improving the curriculum of their respective study programs. However, the impact of staff development programs on improving teaching methods varies among institutions. The Project had no strong focus on improving teaching techniques. Due to a lack of clear pedagogical quality standards, monitoring mechanisms to review lecturers’ performance in the classrooms and workshops were weak. 57. Infrastructure Upgrading. Investments to upgrade facilities—including civil works and procurement of equipment and furniture—greatly helped to improve the overall learning and teaching environment in all participating institutions. However, maintenance of educational equipment and its integration into lessons need to be improved in some institutions to further enhance the learning process. The procurement process was delayed in many cases due to a lack of experience among SPMU staff in preparing equipment specification and bidding documents, and the need to repackage certain study programs from different institutions into one bidding proposal. 58. Study Program Development. In total, 170 study programs have been reviewed and revised at the project institutions. The accreditation level of most study programs supported under the Project was upgraded. Feedback from field visits confirmed that the average grade point average and English proficiency of graduates improved, their employability was enhanced, and the overall completion time of study was reduced. In terms of academic valuation, curriculum review and development was considered highly efficient. However, better industry involvement in the review and development process could have further improved the relevance of study programs. Due to a lack of national standards, the quality of study programs varies among project institutions. 59. Career guidance and professional counseling services in selected institutions are additional examples of providing more efficient support to students seeking employment after graduation. Field visits revealed that the waiting time for recent graduates to find employment was reduced substantially. 60. Retooling programs conducted under the Project were less successful. Originally, the Project provided $4.32 million to retrain about 20,000 unemployed and underemployed graduates in a broad range of skills. In 2004, the allocation for retooling programs was increased to about $13 million ($6.5 million financed by ADB and $6.5 million financed by the Government). On average, about 33% of retrained graduates found employment immediately after completing the training. There was no systematic tracer study conducted to understand the absorption of the remaining graduates into the labor market in subsequent months. Many training providers have developed good linkages with local companies and conducted relevant training courses. Experiences from the retraining programs have not been transferred into the curriculum of the study programs.

15

C. Preliminary Assessment of Sustainability

61. There are significant indications that investments made under the Project are relevant to both Government policy and project institutions, and the Project is therefore expected to be sustainable. Achievements at the policy level are limited but sustainable as the Government is strongly committed to further improve the quality and relevance of higher education. Results of research activities are considered highly relevant and have been incorporated in policy formulation and shared widely within the higher education system. The improvement of the accreditation system for study programs has been accepted as beneficial by employers and the management of higher education institutions. The Government will sustain and further develop the improved accreditation system. As indicated in its current strategic plan, MONE aims to increase the number of study programs rated with either an A or B accreditation from 1,000 to 3,000 between 2005 and 2009. 62. At the institutional level, the project interventions are sustainable. Based on forecasts, the 74 participating higher education institutions will sufficiently allocate budgets to operate and maintain the training equipment that was procured under the Project. However, the utilization of equipment remains a concern. Monitoring reports and field visits revealed that at many institutions the new equipment has not been fully integrated into current lessons, and is underutilized and not well maintained. Lecturers need additional support to prepare training modules that integrate new equipment to enhance practical training opportunities. 63. Investments in staff development are sustainable. Most trained lecturers have returned to their respective higher education institutions where they have contributed to further improvements in the teaching and learning environments. Some higher education institutions were even able to generate income from providing research services to private companies or local governments as a result of the availability of trained staff and modern equipment. 64. The competitive-based funding mechanism has improved transparency, governance, and a sense of ownership among the participating higher education institutions. The Government is committed to further institutionalize this funding mechanism through central budget support. A new competitive grant for higher education institutions was implemented in 2008. 65. Project investments in polytechnics education are sustainable. The Government is committed to further develop and strengthen polytechnics as reflected in the strategic plan of MONE. Promoting polytechnics education is a top priority of the Government in the medium-term development plan and is in accordance with the industry’s growing demand for skilled manpower. 66. Government commitment to improving the higher education system is strong, as reflected by an increasing budget during the last five years. The total budget for the higher education system in 2003 was Rp6.75 trillion, which increased by 11.8% in 2004, 7.6% in 2005, 23.0% in 2006, and 35.0% in 2007. The annual budget for the higher education system in 2007 was Rp12.96 trillion (equivalent to $1.45 billion).14

14 Performance Audit Report of DGHE, December 2007.

16

D. Impact

67. The Project has improved the teaching and learning environment at 74 higher education institutions, which has contributed to the increased mobility and employability of graduates. The major beneficiaries of the Project were about 102,000 students enrolled in the priority study programs of the 74 participating institutions. However, quantifying the benefits for higher education graduates in economic terms is difficult as education by itself is not sufficient to raise incomes. Job opportunities and salary structures depend on various other economic factors. The availability of better quality graduates will have positive impacts on industry and the economy, but it will take time to review the Project’s impact on economic growth. 68. Impacts of the Project at the macro-level are limited. Intended policy reforms and a restructuring of the entire higher education system did not occur because a major reform of the higher education sector had taken place shortly before project commencement. Still, the Project was instrumental in helping DGHE facilitate the implementation of reform strategies. The improved accreditation system is pivotal to increasing the quality and competitiveness of the higher education system. In addition, the implementation of a competitive-based funding mechanism has improved governance and transparency, and stimulated the entrepreneurial spirit of the higher education system country-wide.

69. The Project had no adverse impact on the environment and no resettlement implications. No land acquisition took place during project implementation.

IV. OVERALL ASSESSMENT AND RECOMMENDATIONS

A. Overall Assessment

70. Overall the Project is rated successful. It was well-conceived and -designed, and its overall scope and components remained relevant throughout implementation. The Project has substantially contributed to improving the teaching and learning environments at the 74 project institutions by upgrading physical capacities, improving the accreditation system, strengthening staff development, and improving the relevance and quality of 170 study programs. About 70% of project financing was directed towards improving the teaching and learning environments at the 74 project institutions. While overall progress at these institutions is good, greater attention is now required to ensure adequate utilization of equipment by actively encouraging teaching and learning practices that promote student-centered learning. Future revisions to the study programs should consider increased participation by industry. In addition, curriculum modules should incorporate entrepreneurial principles to prepare students for self-employment and jobs in the informal sector. Scholarships provided under the Project helped almost 20,000 students from low income families to graduate from higher education institutions. 71. The introduction of a competitive selection mechanism to identify higher education institutions to be supported under the Project has enhanced governance and transparency, built a strong of ownership, and ensured the full commitment of the management of higher education institutions. The approach to link subproject grants to business plans, which were developed by the higher education institutions, proved to be successful. While the Project was successful at improving teaching and learning environments at the 74 participating institutions, the potential of the Project to assist DGHE in restructuring the higher education system as formulated in the project objective under Part A has not been fully realized.

17

B. Lessons

72. Major lessons drawn from the Project include: (a) Outcomes and output indicators were not clearly described in order to establish

meaningful monitoring. The design of M&E systems established under the Project did not include quality indicators such as teaching–learning performance, relevance of study programs, and employment rates.

(b) The competitive selection mechanism used to identify institutions for project assistance

created a strong sense of ownership and promoted an entrepreneurial spirit at the institutional level. However, involvement of higher education institutions’ management in supporting the utilization of investments and promoting sustainability varied among institutions.

(c) The staff development program was successful in improving competence and academic

qualifications. However, no systematic mechanism was in place to strengthen teaching methods after the participating staff returned to their respective universities.

(d) Integration of curriculum development and teaching–learning methodologies, and the

application of modern lab equipment have not been put into place. The utilization and maintenance of new equipment is not adequately monitored by the management of higher education institutions.

(e) Curriculum revision of similar study programs was done by higher education institutions

with limited coordination, leading to duplication of efforts in many areas. Industry participation varied from one higher education institution to the next.

(f) Although the Project provided hands-on training on procurement procedures to project

staff at the SPMUs, significant delays in the bidding process and fraudulent procurement practices still occurred.

(g) The Project was overloaded with reporting requirements (e.g., monthly, quarterly, annual,

and end-of project reports) from each of the SPMUs, while the CPMU had limited resources to review, analyze, and provide feedback on the reports.

C. Recommendations

73. Recommendations for project design and implementation include:

(a) Outcome indicators need to be clearly described in order to establish meaningful monitoring. At the institutional level, tracer mechanisms need to be improved to systematically collect information on the employment rates of graduates, which would be useful to strengthen the relevance of study programs; and the design of M&E systems needs to be improved to allow for the collection of qualitative information on teaching performance, relevance of study programs, and employment rates.

(b) For future investment, competitive-based selection needs to be continued and improved

upon by placing greater emphasis on the commitment and capacity of higher education management, as well as on labor market information relevant to the graduates of assisted study programs. The commitment and involvement of higher education

18

management in institutionalizing innovations initiated by the Project are critical for the sustainability of project investments.

(c) The higher education institutions need to provide further support and guidance for

trained staff to improve their professionalism in the teaching–learning practice. Many institutions have developed quality assurance centers and teaching–learning centers that require continued commitment and support from their respective management. DGHE needs to encourage such institutions to share their experiences and lessons learned with other institutions through workshops and seminars.

(d) To ensure the proper use of equipment and its integration into the curriculum, monitoring

mechanisms need to be improved to inform higher education management and DGHE of utilization status. Lecturers need additional support to prepare training modules that integrate new equipment to enhance practical training opportunities.

(e) National standards should be recognized as the starting point for curriculum

development and revision. To ensure that the quality and relevance of study programs are strengthened throughout the higher education system, program development should be driven by national competency and skills standards and guided by labor market information. Industry involvement needs to be increased at all levels.

(f) In addition to hands-on training on procurement procedures for project staff, continued

technical guidance and better monitoring and recording systems should be implemented by the central management to minimize delays and noncompliance.

(g) A simplified reporting and monitoring system needs to be designed to ensure that reporting requirements can be reduced. At the same time, the progress and obstacles faced by each sub-project should be monitored in a timely manner. In addition, the benefit monitoring and evaluation system should be in place to provide meaningful information on the outputs, outcomes, and benefits of the Project. The design of M&E systems needs to be improved to collect qualitative information on teaching performance, relevance of study programs, and employment rates.

(h) The EA needs to ensure that the ongoing construction and rehabilitation of buildings at

two universities in Aceh, which were only 60% finished at the project completion date, will be completed in 2008 with funding from the Government.

Appendix 1 19

LIST OF HIGHER EDUCATION INSTITUTIONS (As Completed)

No. Name of HE Institution Address Phone

Fax Region I 1 Politeknik Negeri Lhokseumawe Jl. Banda Aceh-Medan KM. 280, Buketrata, Lhokseumawe 0645 42 785 0645 42 785 2 Universitas Iskandar Muda Jl. Kampus UNIDA Surien, Banda Aceh 0651 42 908 0651 42 225 3 Universitas Syiah Kuala FMIPA UNSYIAH, Darussalam, Banda Aceh 0651 53 774 0651 7411129 4 Institut Teknologi Medan Jl. Gedung Arca No. 52, Medan 061 736 3771 061 734 7954 5 Politeknik Negeri Medan Jl. Almamater No.1 Kampus USU, Medan 061 821 0371 061 821 5845 6 Universitas Muhammadiyah Sumatera

Utara Gedung Fakultas Teknik UMSU Lt.2 Jl. Demak No. 3, Medan 061 736 1515 061 662 5474

7 Universitas Negeri Medan Jl. Willem Iskandar, Pasar V, Medan 061 664 1347 061 661 3319 8 Universitas Sumatera Utara Jl. Dr. Mansur No. 9, Kampus USU, Medan 061 821 1049 061 8211049 9 Universitas Riau Kampus Bina Widya, KM 12.5 SP Panam, Pekanbaru 0761 63 226 0761 63 279

10 Institut Teknologi Padang15 Jl. Gajah Mada Kandis Nanggalo, Padang 0751 444 842 0751 444 841 11 Politeknik Negeri Padang Kampus Limau Manis, Padang 0751 72 576 0751 72 590 12 Politeknik Pertanian Negeri Payakumbuh Tanjung Pati, Kabupaten Limapuluh Kota 0752 92004 0752 50220 13 Universitas Andalas Gd. Rektorat Lt. II Kampus UNAND Limau Manis, Padang 0751 777 290 0751 777 290 14 Universitas Bung Hatta Jl. Gajah Mada No. 19, Padang 25143 0751 705 4257 0751 51 341 15 Universitas Eka Sakti Kampus Univ. Eka Sakti Jl. Veteran Dlm No. 26B, Padang 0751 26 770 0751 32 694 16 Universitas Negeri Padang Jl. Prof. Dr. Hamka, Air Tawar, Padang 0751 445 998 0751 55 644 17 Universitas Putra Indonesia "YPTK"16 Jl. Raya Lubuk Linggau Begalung, Padang 0751 776 666 0751 71 913 18 Politeknik Manufaktur Timah Bangka Jl. Air Kantung Sungailiat, Bangka 0717 93 586 0717 93 585 19 Politeknik Negeri Sriwijaya Jl.Srijaya Negara, Palembang 0711 376 363 0711 376 364 20 Universitas Binadarma17 Jl. A. Yani No. 3, Palembang 0711 518000 0711 518000 21 Universitas Muhammadiyah Palembang Jl. Jend. A. Yani 13 ULU, Palembang 0711 518 394 0711 518 394 22 Universitas Sriwijaya Jl. Raya Palembang-Prabumulih Km 32 Inderalaya, Palembang 0711 581 010 0711 581010 23 STMIK Darmajaya Bandar Lampung Jl. Zainal Abidin Pagar Alam No. 93 , Bandar Lampung 0721 700 261 0721 700 261 24 Universitas Bandar Lampung Jl. Zainal Abidin Pagar Alam No. 26 , Bandar Lampung 0721 771 331 0721 701 467 25 Universitas Lampung Jl. Soemantri Brojonegoro No. 1, Bandar Lampung 0721 783 682 0721 783 682

15 Sekolah Tinggi Teknologi Padang upgraded into Institut Teknologi Padang in 2002 16 STMIK Padang upgraded into Universitas Putra Indonesia “YPTK” Padang in 2003 17 STMIK Bina Darma upgraded into Universitas Bina Darma in 2002.

20

Appendix 1

No. Name of HE Institution Address Phone Fax

26 Universitas Tridinanti Jl. Kapten Marzuki No. 2446 Komboja, Palembang 0711 357526 0711 358566 27 Universitas Tanjungpura Komp. Rektorat UNTAN Jl. A. Yani, Pontianak 0561 739 636 0561 739636 28 Politeknik Negeri Pontianak Jl. Ahmad Yani, Pontianak 0561 736 180 0561 740 143

Region II 29 Institut Teknologi Indonesia Jl. Raya Puspitek Serpong, Kampus ITI , Tangerang 021 756 1645 021 756 1645 30 Universitas Bina Nusantara Jl. Kebun Jeruk Raya No. 27 Kemanggisan, Jakarta Barat 021 536 96921 021 530 0244 31 Universitas Gunadarma Jl. Margonda Raya No. 100, Pondok Cina, Depok 021 78881112 021 7872829 32 Universitas Muhammadiyah Jakarta Jl. Cempaka Putih Tengah No. 27, Jakarta 021 4256024 021 4256023 33 Institut Teknologi Nasional Jl. PHH Mustafa No. 23, Bandung 022 720 8067 022 720 1812 34 Politeknik Manufaktur Bandung Jl. Kanayakan No. 21, Bandung 40135 022 250 0241 022 250 2649 35 Politeknik Negeri Bandung Jl. Gegerkalong Hilir – Desa Ciwaruga, PO. Box 6468, Bandung 022 201 4529 022 201 4529 36 Universitas Padjadjaran Jl. Raya Bandung - Sumedang KM 21, Jatinangor, Bandung 022 779 6014 022 779 6014 37 Universitas Diponegoro Jl. Prof. Soedarto, SH. Tembalang, Semarang 024 747 4770 024 746 0060 38 Universitas Jenderal Soedirman Jl. Prof. Dr. HR. Boejamin 708 Kotak Pos 15, Purwokerto 0281 628 709 0281 628 709 39 Universitas Islam Indonesia Jl. Cik Ditiro No. 1, Yogyakarta 0274 895 042 0274 895 330 40 Universitas Muhammadiyah Yogyakarta Jl. Lingkar Selatan , Jl. HOS Cokroaminoto 17, Yogyakarta 0274 618 363 0274 618 363 41 Universitas Negeri Yogyakarta Kampus UNY, Karang Malang, Yogyakarta 0274 540 715 0274 540 715 42 Universitas Muhammadiyah Surakarta Jl. Ahmad Yani Tromol Pos I Pabelan, Kartasura, Surakarta 0271 725 084 0271 725 084 43 Universitas Sebelas Maret Jl. Ir. Sutami No. 36A Kentingan, Surakarta 0271 668 603 0271 667 887 44 Institut Teknologi Sepuluh November Kampus ITS Sukolilo, Gd. Rektorat Lt. 1, Surabaya 031 592 1586 031 592 1586 45 Politeknik Perkapalan Negeri Surabaya Jl. Teknik Kimia, Kampus ITS Sukolilo, Surabaya 031 592 3624 031 592 5524 46 Universitas Airlangga Jl. Airlangga No. 4-6, Surabaya 031 504 4871 47 Universitas Katolik Widya Mandala Jl. Kalijudan 37, Surabaya 031 3891264 031 389 1267 48 Universitas Kristen Petra Surabaya Jl. Siwalan Kerto 121-131, Surabaya 031 843 9040 031 849 1215 49 Universitas Surabaya Jl. Ngagel Selatan 169, Surabaya 031 298 1008 031 5029736 50 Politeknik Negeri Malang Jl. Veteran, PO. BOX 04, Malang 0341 551 340 0341 551 708 51 Universitas Brawijaya Jl. May Jen Haryono 169, Malang 0341 575 801 0341 575 801 52 Universitas Jember Jl. Kalimantan 37, Jember 0331 332 735 0331 332 739 53 Universitas Negeri Malang Jl. Surabaya No. 6, Malang 0341 587 927 0341 587 927

Region III 54 Politeknik Negeri Bali Bukit Jimbaran, Badung 0361 703 212 0361 703 212 55 Universitas Udayana Gedung Rektorat Lantai III, Kampus Bukit Jimbaran 0361 704 846 0361 704 846 56 STMIK Bumigora, Mataram Jl. Ismail Marzuki, Mataram 0370 636 898 0370 638 369 57 Universitas Islam Al-Azhar Jl. Unizar No. 20 Turida Cakranegara, Mataram 0370 671 650 0370 671 650 58 Universitas Mataram Jl. Majapahit 62, Mataram 0370 634 783 0370 634 784

Appendix 1

21

No. Name of HE Institution Address Phone Fax

59 Politeknik Pertanian Negeri Kupang Jl. Adi Sucipto, Penfui, Kupang 0380 882 029 0380 881 601 60 Politeknik Negeri Ujung Pandang Jl. Perintis Kemerdekaan KM. 10 Tamalanrea, Makasar 0411 585 368 0411 586 043 61 Politeknik Pertanian Negeri Pangkep Jl. Poros Makssar - Pare Pare KM. 83 Seger Mandalie, Pangkep 0411 581 588 0411 583 336 62 Universitas Atmajaya Makassar Jl.Tanjung Alang No. 23, Makasar 0411 859651, 0411 859651 63 Universitas Hasanuddin Jl. Perintis Kemerdekaan, Kampus Tamalanrea, Makasar 0411 586028 0411 585188 64 Universitas Kristen Indonesia Paulus Jl. Perintis Kemerdekaan KM. 13 Daya, Makasar 0411 586 042 0411 586 042 65 Universitas Muslim Indonesia Jl. Urip Sumoharjo KM-05, Kampus II UMI, Makasar 0411 425 576 0411 425 576 66 Universitas Negeri Makassar Kampus Fakultas Teknik, UNM, Parang Tambung, Makasar 0411 840 894 0411 868 879 67 Universitas Haluoleo Kampus Bumi Tridharma, Jl. May Jen S. Parman, Kendari 0401 322763 0401 395530 68 Universitas Tadulako Kampus Bumi Tadulako Tondo, Palu 0451 422 611 0451 422844 69 Universitas Sam Ratulangi Kampus UNSRAT, Bahu, Manado 0431 850 634 0431 850 634 70 Universitas Negeri Manado Kampus UNIMA di Tondano, Manado 0431 321 845 0431 321 517 71 Politeknik Negeri Manado Kampus Politeknik, Desa Bahu, Manado 72 Politeknik Negeri Samarinda Jl. Dr. Cipto Mangunkusuma, Kampus, Gunung Lipan,

Samarinda 0541 260 588 0541 260 355

73 Universitas Lambung Mangkurat Jl. Brigjen H. Hasan Basry, Gedung Rektorat Lt. 3, Banjarmasin 0511 306 614 0511 306 614 74 Universitas Sains dan Teknologi

Jayapura 18 Jl. Raya Sentani, Padang Bulan Abepura, Jayapura 0967 587 403 0967 587 403

STMIK = Sekolah Tinggi Manajemen Informatika dan Komputer (Higher Education on Management Informatics and Computer) Note: a. Region I covered the HE in Western Part of Indonesia (Sumatra Island and West Kalimantan Province); . Region II covered the HE in Java; c. Region III covered the HE in Eastern Part of Indonesia (the remaining HE) Sources: Directorate General of Higher Education 2007

18 Institut Sains dan Teknologi Jayapura upgraded into Universitas Sains dan Teknologi Jayapura in 2003.

22 Appendix 2

OUTPUT TARGETS AND ACHIEVEMENTS

Design Summary

Indicator and Targets

Achievements

Goal Improved international competitiveness and help to promote sustainable economic growth that contributes to poverty reduction on a gender, social, and geographically equitable basis

• Successful implementation of the

Government ‘s long-term higher education strategy

• Increased mobility and employability of graduates from the higher education system

• Increased productivity and wages for higher education graduates, including women and economically disadvantaged groups

• Government committed to implement

long-term higher education strategy • Increased mobility and employability of

graduates from the higher education system

• Increased wages for higher education graduates, including women and economically disadvantaged groups

Purpose Increased efficiency and effectiveness of higher education institutions on a regional basis through improved quality, capacity, and relevance of priority disciplines in the public and private higher education system

• Waiting time to find a job after

graduation reduced by 20% • 75% of graduates find discipline-

relevant work within 12 months of graduation

• Starting salaries for graduates increased

• Participation by women increased to a mean of 50% in the priority disciplines

• Participation by the economically disadvantaged increased by 20,000 students in the priority disciplines

• Participating public universities move closer to becoming autonomous institutions

• Closer partnership between public and private institutions of higher education established

• Accreditation level of 20 study

programs upgraded from B to A; 2 study programs from C to A; 23 study programs from C to B; and 9, 22, and 1 study programs from un-accredited to A, B, and C, respectively

• Length of study of graduates in higher education institutions reduced by 13% on average

• English proficiency of students increased by 15% (institutional Test of English as a Foreign Language [TOEFL] test)

• Job waiting time after graduation reduced by 13% on average

• 75 % of graduates find work within 12 months after graduation

• Starting salaries for graduates increased on average by 20%

• Participation of women in priority disciplines increased by 30 %

• 18,674 students from low income families have been supported by grants

• 8 public universities prepared for autonomous status (Unair, Polman, Polban, Unhas, Unsri, Undip, Unibraw and Universitas Sumatera Utara)

• Network between public and private institutions establish in Padang, Medan, Yogya, Palembang, Makassar, Bandung, Mataram and Jakarta.

• Employability skills training delivered to all graduates and about 20,000 unemployed and underemployed graduates in the priority disciplines

• Community satisfaction levels increased

• Local (institutional) budgeted contribution adequate

• 14,671 unemployed graduates trained • Community satisfaction levels

increased by 16 % as demonstrated by increased applications to project study programs.

• Counterpart budget contribution adequate throughout the project

Appendix 2

23

Design Summary

Indicator and Targets

Achievements

Project Component and Output Part A : 1. Strengthened

management capacity for development and sustainability of higher education institutions

2. Strengthened National

Accreditation Agency (BAN) and Board of Higher Education (BHE)

• Managers and administrators

trained in financial and education management

• The project management information system (MIS) operational at all participating campuses and Directorate General of Higher Education (DGHE)

• BAN accreditation capacity increased by 50% to accelerate the application of quality assurance and quality control procedures across the higher education system

• BHE strengthened through institutional and study program development, logistic support, and research and development activities

• Engineering Quality Assurance Council procedures adapted and applied to polytechnic programs

• The project benefit monitoring and

evaluation system has been developed at the central project management unit and sub-project management unit levels. However, the benefit and monitoring evaluation system is not fully integrated in the management information system of DGHE.

• BAN capacity increased; all project

institutions are accredited by BAN. • BHE provided experts and reviewers

to review progress • DGHE quality assurance has been

developed, with references from Project studies

3. Improved Equity 4. Retrained Graduates

• 20,000 grants and scholarships awarded, 50% to women to enhance access for the economically disadvantaged

• 20,000 unemployed and

underemployed graduates retrained in four regions through short term skills training priority study programs

• 18,674 grants and scholarships awarded, 41.8% to women to enhance access for the economically disadvantaged

• 14,671 un employed and

underemployed graduates retrained in three regions through short term skill training priority study programs

24 Appendix 2

Design Summary

Indicator and Targets

Achievements

Part B: Strengthened priority disciplines and new study programs developed

• Academic staff upgraded through international and domestic training, including on-the-job training

• 22 diploma (3 years) graduates upgraded to diploma (4 years) graduates to become polytechnic instructors

• About 130 study programs in basic science, engineering, agriculture, law, management, and accounting upgraded

• Lecture and lab course revised to enhance skill and competency content

• New course development and delivered using new information technology –based instruction methods

• 4,046 academic staff upgraded through international and domestic training, including on-the-job training

• 18 diploma (3 years) graduates upgraded to diploma (4 years) graduates to become polytechnic instructors

• 170 study programs in basic science, engineering, agriculture, law, management, and accounting upgraded

• 803 lecture and lab course revised to enhance skill and competency content under teaching research grants

• 7 new courses developed and delivered using new information technology –based instruction methods

• 3,133 workshops conducted to share lesson learned and strengthen study program development

• 1,345 research grants, 1,360 teaching grants, and 3,032 student research grants awarded

• Class and lecture rooms, and labs

and shops renovated • About 30 new study programs in

basic science, engineering, agriculture, law, and management and accounting established with upgraded supporting facilities

• 355 packages contracted to renovate classrooms, lecture rooms, labs and workshops

• 12 new study programs in basic science, engineering, agriculture, management, and accounting established with upgraded supporting facilities

• At least 6 study centers for women

strengthened to promote equitable participation of woman in higher education

• Teaching learning centers establishes

• Quality assurance centers established

• 6 women study centers strengthened

to promote equitable participation of woman in higher education

• 20 teaching and learning centers established

• 9 quality assurance centers established

• Computer laboratories and English language laboratories established or upgraded

• Institutional MIS and information technology network established, including online library facilities and integration of the benefit and monitoring evaluation system

• 8 computer and information technology networks established

• 14 English language laboratories established

• 15 institutional MIS established • 11 online library facilities established • 13 central library facilities established

• Employers liaison board created or strengthened at participating institutions

• Community service centers created and functioning

• Student advisory, job placement,

• 2 community business services created, including employers liaison board

• 12 student advisory, job placement,

Appendix 2

25

Design Summary

Indicator and Targets

Achievements

and carrier development centers created and functioning efficiently

and carrier development centers created

Additional Component 1. Aceh Support Program 2. Yogyakarta Support Program

• Project institution buildings affected

by the tsunami and earthquake in Aceh constructed and renovated

• Scholarships for students affected by disaster in Aceh

• Consulting services to support educational activities in Universitas Syiah Kuala

• Education facilities affected by

earthquake rehabilitated and replaced

• Scholarships for students affected by earthquake in Yogyakarta

• Reconstruction of infrastructure at

University of Syiah Kuala and Universitas Iskandar Muda has started and will be completed by the Government

• 488 grants and scholarship awarded under the Aceh support program, 34 % to women

• Rehabilitation of education facilities

completed • 600 grants and scholarship awarded

for Yogyakarta support program, 45 % to women

BAN = badan akreditasi nasional (national accreditation agency), BHE = board of higher education, DGHE = directorate general of higher education Source: Asian Development Bank estimates.

26 Appendix 3

NUMBER OF PROPOSALS AND AWARDED GRANTS (batch I-III and the distribution among region I, II, and III)

Submitted Proposal Awarded Region Region

Batch

Program I II III Total I II III Total

Strengthening Study Program 111 265 110 486 12 10 8 30 New Study Program 18 39 25 82 — — 3 3 Institutional Support Services 44 71 37 152 1 2 2 5

Batch I Subtotal 173 375 172 720 13 12 13 38

Strengthening Study Program 95 231 78 404 20 17 18 55 New Study Program 11 8 17 36 — 1 — 1 Institutional Support Services 12 46 18 76 3 9 — 12

Batch II Subtotal 118 285 113 516 23 27 18 68

Strengthening Study Program 77 184 61 322 34 19 20 73 New Study Program 2 6 18 26 1 — 7 8 Institutional Support Services 31 43 24 98 11 21 8 40

Batch III Subtotal 110 233 103 446 46 40 35 121

Additional Institutional Support Services

52 78 53 183 12 30 11 53

Strengthening Study Program 283 680 249 1,212 66 46 46 158 New Study Program 31 53 60 144 1 1 10 12 Institutional Support Services 139 238 132 509 27 62 21 110

Total

453 971 441 1,865 94 109 77 280 —= no data available Note: The grants for study programs (SP) and institutional support services (ISS) were contracted to higher education institutions in three batches. Contract for Batch I: Jan 2002 – Mar 2006; Batch—2: Jan 2003 – June 07; Batch III: Jan 04 – Dec 07. In 2006 the EA awarded additional contracts for ISS in 74 higher education institutions, primarily to support the development and strengthening of Teaching—learning Center, Woman Study Center, Community and Industrial Relation and Quality Assurance Units. Source(s): Directorate General of Higher Education estimates  

Appendix 4 27

SUMMARY OF RETOOLING PROGRAMS

Batch I – 2003 Batch II – 2004 Field of Study No. of Class Graduates No. of Class Graduates Employed % Engineering 19 950 62 2,988 737 25 Basic Sciences 2 100 10 428 126 29 Agriculture 6 300 21 876 227 26 Economics 4 197 17 714 200 28 Law 2 98 4 194 64 33 Entrepreneurships _ _ 5 233 151 65 Total 33 1,645 119 5,433 1,505 27.70%

Batch III - 2005 Batch IV - 2006

Field of Study No. of Class Graduates Employed % No. of Class Graduates Employed %

Engineering 46 2,110 753 35.69 63 2,928 911 31.11% Basic Sciences 6 232 87 37.50 5 234 92 39.32% Agriculture 11 461 228 49.46 15 717 253 35.29% Economics 7 275 99 36.53 3 134 N/A N/A Law 3 137 41 29.93 3 143 27 18.88% Entrepreneurships 5 222 81 36.49 — — — Total 78 3,437 1,289 37.50% 89 4,156 1,283 30.90%

—= data not available Note: The employed numbers were participants have a job right after the training completion of each batch. Source: Reports on Retooling Program Implementation 2003 to 2006

28 Appendix 5

SUMMARY OF STAFF DEVELOPMENT

Overseas Degree Program Domestic Degree Program Discipline Planned by

Study Program

Awarded by the Project

Completed OG Planned by Study Program

Awarded by the Project Completed OG

D4 S2 S3 D4 S2 S3 D4 S2 S3 D4 S1 S2 S3 D4 S1 S2 S3 D4 S1 S2 S3 Basic Science 0 50 24 0 19 42 0 18 15 27 0 0 165 70 0 0 8 18 0 0 8 2 15Engineering 0 110 35 0 137 112 0 125 43 73 40 33 410 95 21 0 219 34 21 0 14 14 44Agriculture 0 25 12 0 12 21 0 11 12 9 40 0 105 25 0 0 14 21 0 0 36 11 10Economics 0 25 0 0 24 31 0 23 14 16 18 0 115 0 0 0 36 46 0 0 7 14 32Law 0 10 10 0 4 5 0 4 1 4 0 0 30 0 0 0 7 10 0 0 0 7 3Derivatives 0 0 0 0 27 0 0 21 0 3 0 0 0 0 0 0 69 1 0 0 52 0 17ISS 0 0 0 0 6 0 0 3 0 0 0 0 0 0 5 0 0 0 5 0 0 0 0

Total 0 220 81 0 229 211 0 205 85 132 98 33 825 190 26 0 353 130 26 0 117 48 121OG = on going (means continued study funded by the government) Notes: a. D4 means instructor for polytechnic; b. S1 means bachelor degree; c. S2 means master degree; d. S3 means doctorate degree; e. 8 fellows for overseas master degree, 7 fellows of overseas and 3 fellows of domestic doctorate degree respectively were drop out Source: Report on Staff Development Program 2007

Appendix 6 29

ALLOCATION OF STAFF DEVELOPMENT BY HIGHER EDUCATION INSTITUTIONS Staff Development Overs. Degree OND Dom.Degree DND Target Actual Target Actual Target Actual Target Actual No HE Institution Person Person Person Person Person Person Person Person

1 Inst. Tekn Indonesia 9 8 22 9 4 4 50 29

2 Inst. Tekn Medan 4 — 2

— 10 10 11 7

3 Inst. Tekn Nasional 14 14 27 21 5 4 61 46 4 Inst. Tekn Padang 2 2 13 4 14 14 92 90 5 Inst. Tekn Sepuluh Nopember 18 18 32 28 10 10 204 163 6 Poli Manufaktur Bandung 1 1 7 7 4 4 16 10 7 Poli Manufaktur Timah Bangka 9 8 17 13 17 17 62 37 8 Poli Negeri Bali 13 13 43 36 14 13 81 75 9 Poli Negeri Bandung 7 7 88 85 15 15 170 169

10 Poli Negeri Lhokseumawe 4 3 22 4 4 4 65 40 11 Poli Negeri Malang 4 4 35 33 13 12 56 31 12 Poli Negeri Medan 4 4 13 7 12 12 50 24 13 Poli Mnegeri Menado 14 Poli Negeri Padang 10 7 18 1 19 18 100 79

15 Poli Negeri Perkapalan 3 — 47 6 9 8 47 110

16 Poli Negeri Pontianak 3 3 6 4 10 9 22 12

17 Poli Negeri Samarinda —

— 12 8 12 8 30 26

18 Poli Negeri Sriwijaya 9 8 20 16 39 38 334 332 19 Poli Negeri Ujung Pandang 10 6 14 7 23 19 94 73

20 Poli Tani Kupang 1 1 5 — 2 2 20 15

21 Poli Tani Pangkep —

— 6 1

—

— 70 57

22 Poli Tani Payakumbuh 3 1 15 8

23 STMIK Bumigora 1 1 3 — 8 7 20 18

24 STMIK Darmajaya 6 6 10 4 12 12 29 29 25 Univ Airlangga 5 5 25 23 5 5 31 26 26 Univ Andalas 19 18 46 29 8 8 49 43

27 Univ Atmajaya Mks 4 3 —

— 1 1 2

—

28 Univ Bandar Lampung 4 3 4 — 6 6 50 46

29 Univ Bina Nusantara 9 9 21 13 5 5 192 162 30 Univ Binadarma 7 7 5 5 3 3 47 47 31 Univ Brawijaya 6 6 38 31 5 3 61 55

32 Univ Bung Hatta 3 3 16 8 —

— 24 17

33 Univ Diponegoro 17 17 36 17 3 3 22 17

34 Univ Ekasakti 2 1 7 5 —

— 3

—

35 Univ Gunadarma 9 9 18 18 3 1 11 20 36 Univ Haluoleo 5 4 8 1 11 11 28 28 37 Univ Hasanuddin 11 10 34 23 7 6 92 57

30 Appendix 6

38 Univ Iskandar Muda 5 3 9 2

39 Univ Islam Al Azhar 4 — 2

— 6 6 6 4

40 Univ Islam Indonesia 4 4 17 4 6 3 57 31 41 Univ Jember 4 4 8 7 10 10 38 25 42 Univ Jenderal Soedirman 6 5 14 5 4 4 24 12

43 Univ Katholik Widya Mandala 3 3 21 16 — 2 2

44 Univ Kristen Paulus 6 4 5 4 8 8 45 28

45 Univ Kristen Petra 10 10 15 11 —

— 80 77

46 Univ Lambung Mangkurat 14 13 13 5 10 10 55 39 47 Univ Lampung 15 14 22 15 19 17 62 51 48 Univ Mataram 10 9 20 11 20 18 83 79 49 Univ Muhammadiyah Jakarta 4 6 5 12 4 3 21 28

50 Univ Muhammadiyah Palembang 4 4 5 — 2 2 15 5

51 Univ Muhammadiyah Sumatera Utara

—

—

—

— 15 15 28 4

52 Univ Muhammadiyah Surakarta 24 24 17 7 13 12 42 25

53 Univ Muhammadiyah Yogyakarta 3 3 7 1 —

— 28 16

54 Univ Muslim Indonesia 1 1 13 7 12 7 29 26

55 Univ Negeri Makassar 2 2 6 3 —

— 24 27

56 Univ Negeri Malang 5 1 10 9 9 6 51 36 57 Univ Negeri Manado 3 2 7 1 3 2 6 6

58 Univ Negeri Medan —

— 4 3 4 4 31 26

59 Univ Negeri Padang 2 2 8 3 2 2 37 33

60 Univ Negeri Yogyakarta —

— 7

— 6 6 65 41

61 Univ Padjajaran 1 1 11 8 3 2 19 13

62 Univ Putra Indonesia 2 1 —

—

—

— 6

—

63 Univ Riau 6 4 22 5 7 8 39 19

64 Univ Sains dan Tekn Jpura 2 1 24 — 17 16 53 47

65 Univ Sam Ratulangi 7 6 20 8 11 11 35 24 66 Univ Sebelas Maret 3 3 9 2 2 2 13 9 67 Univ Sriwijaya 15 13 42 27 11 11 83 77 68 Univ Sumatera Utara 7 6 57 36 11 10 54 38 69 Univ Surabaya 11 10 47 43 3 3 128 127 70 Univ Syiah Kuala 26 24 45 31 6 6 70 62 71 Univ Tadulako 9 9 14 5 15 13 46 35 72 Univ Tanjungpura 11 11 3 1 7 7 73 65 73 Univ Tridinanti 2 2 5 374 Univ Udayana 14 13 30 21 30 29 122 89 471 422 1,273 779 596 550 3,895 3,229 DND = domestic non degree, OND = overseas non degree Note: Target means proposed by each institution Source: Reports on Staff Development Program 2007

Appendix 7 31

STUDY PROGRAM ACCREDITATION 2008

Accreditation Institution Study Program Prop Original Current

Institut Teknologi Indonesia Chemical Engineering SSP B B Electrical Engineering SSP B B Institut Teknologi Medan Mechanical Engineering SSP C C Institut Teknologi Nasional Bandung Chemical Engineering SSP B B Civil Engineering SSP B B Industrial Engineering SSP B A Mechanical Engineering SSP B B Institut Teknologi Padang Civil Engineering (D3) SSP C C Mechanical Engineering (D3) SSP B B Institut Teknologi Sepuluh Nopember Chemistry SSP B A Civil Engineering SSP A A Civil Engineering (D3) SSP NA B Electrical Engineering SSP A A Mechanical Engineering (D3) SSP NA NA Politeknik Manufaktur Bandung Foundry Engineering SSP A A Politeknik Manufaktur Timah Bangka Electronic Technology SSP NA A Machine Maintenance Technology SSP NA B Politeknik Negeri Bali Accounting SSP NA A Business Administration SSP NA A Refrigeration and Air Conditioning NSP NA NA Travel and Tourism SSP NA B Politeknik Negeri Bandung Marketing NSP NA NA Mechanical Engineering SSP A A Refrigeration and Air Conditioning SSP C B Politeknik Negeri Lhokseumawe Mechanical Engineering SSP NA B Telecommunication Engineering SSP NA B Politeknik Negeri Malang Accounting SSP B A Business Administration and Secretary SSP B B Civil Engineering SSP B A Electrical Engineering SSP NA A Politeknik Negeri Manado Accounting SSP NA B Marketing NSP NA B Underwater Ecotourism Department NSP NA N/A Politeknik Negeri Medan Business Administration SSP NA B Civil Engineering SSP NA B Electrical Engineering SSP NA B Mechanical Engineering SSP NA B Politeknik Negeri Padang Civil Construction SSP NA B Electronics Engineering SSP NA B Mechanical Engineering SSP NA C Politeknik Negeri Pontianak Accounting SSP NA B Electronics Engineering NSP NA NA Politeknik Negeri Samarinda Accounting SSP NA A Business Administration SSP NA B

32 Appendix 7

Politeknik Negeri Sriwijaya Business Administration SSP NA A Chemical Engineering SSP NA B Civil Engineering SSP NA A Electronics Engineering SSP NA B Mechanical Engineering SSP NA B Politeknik Negeri Ujung Pandang Accounting SSP C B Chemical Engineering SSP B B Energy Convertion SSP C C Mechanical Engineering SSP C B Telecommunication Engineering SSP B B Politeknik Perkapalan Negeri Surabaya Marine Engineering SSP NA A Ship Design and Construction SSP NA A Politeknik Pertanian Negeri Kupang Animal and Health NSP NA NA Politeknik Pertanian Negeri Pangkep Fishing Technology SSP NA NA Politeknik Pertanian Negeri Payakumbuh Food Technology SSP NA B STMIK Bumigora Mataram Technique of Informatics SSP C B STMIK Darmajaya Bandar Lampung Computer Science SSP C C Universitas Airlangga Management SSP B A Universitas Andalas Agricultural Engineering SSP B B Biology SSP A A Chemistry SSP B A Law (International Law) SSP A A Management SSP B A Universitas Atmajaya Makassar Electrical Engineering SSP B B Universitas Bandar Lampung Civil Engineering SSP NA B Universitas Bina Darma Computing Science SSP B B Universitas Bina Nusantara Computer Science SSP B B Universitas Brawijaya Biology SSP B A Physics SSP B B Universitas Bung Hatta Civil Engineering SSP B B Mechanical Engineering SSP B B Universitas Diponegoro Accounting SSP A A Chemical Engineering SSP A A Civil Engineering SSP A A Universitas Eka Sakti, Padang Civil Engineering SSP C C Universitas Gunadarma Computer System Engineering SSP A A Universitas Haluoleo Civil Engineering (D3) SSP C C Management SSP B B Universitas Hasanuddin Agricultural Engineering SSP C B Agricultural Product Technology SSP B B Chemistry SSP C A Civil Engineering SSP A A Electrical Engineering SSP B B Universitas Iskandar Muda, Banda Aceh Civil Engineering SSP C C Universitas Islam Al-Azhar, Mataram Civil Engineering SSP C C Universitas Islam Indonesia Civil Engineering SSP A A Universitas Jember Law (Economic Law) SSP B A Management SSP B A

Appendix 7 33

Universitas Jenderal Soedirman Plant Breeding SSP B B Universitas Katolik Widya Mandala Surabaya Chemical Engineering SSP A B Universitas Kristen Indonesia Paulus, Ujung Pandang Civil Engineering SSP C B Mechanical Engineering SSP B B

Telecommunication and Electronics Engineering SSP C C

Universitas Kristen Petra Surabaya Mechanical Engineering SSP A B Universitas Lambung Mangkurat Accounting SSP B B Chemical Engineering NSP NA NA Civil Engineering SSP B B Management SSP B B Physics NSP NA NA Universitas Lampung Accounting SSP B B Agricultural Engineering SSP C B Law (Economic) SSP B A Management SSP B A Post Harvest Technology SSP B B Universitas Mataram Animal Product Processing NSP NA NA Law (Business Law) SSP C B Management SSP C B Plant Breeding SSP C C Universitas Muhammadiyah Jakarta Chemical Engineering SSP B B Universitas Muhammadiyah Palembang Chemical Engineering SSP C C Universitas Muhammadiyah Sumatera Utara Civil Engineering SSP B B Universitas Muhammadiyah Surakarta Chemical Engineering SSP B B Civil Engineering SSP B A Electrical Engineering SSP B B Mechanical Engineering SSP B B Universitas Muhammadiyah Yogyakarta Mechanical Engineering SSP B B Universitas Muslim Indonesia, Ujung Pandang Chemical Engineering SSP B B Electrical Engineering SSP C B Mechanical Engineering SSP C B Universitas Negeri Makassar Electronics Engineering SSP C B Universitas Negeri Malang Civil Engineering SSP NA B Universitas Negeri Manado Wood Technology NSP NA NA Universitas Negeri Medan Mechanical Engineering SSP C C Universitas Negeri Padang Electrical Engineering SSP B B Universitas Negeri Yogyakarta Electrical Engineering SSP B C Universitas Padjadjaran Physics SSP A A Universitas Putra Indonesia "YPTK" Computer Engineering SSP C B Universitas Riau, Pekanbaru Biology SSP C B Management SSP B B Universitas Sains dan Teknologi Jayapura Civil Engineering SSP C C Electrical Engineering SSP C B Mechanical Engineering SSP C B Universitas Sam Ratulangi Agricultural Engineering SSP C B

34 Appendix 7

Agricultural Product Technology SSP B A Civil Engineering SSP B A Management SSP B A Universitas Sebelas Maret, Surakarta Accounting SSP A A Universitas Sriwijaya Biology SSP B A Chemical Engineering SSP B B Civil Engineering SSP A A Law (Economic Law) SSP A A Physics SSP B B Universitas Sumatera Utara Accounting SSP B B Chemistry SSP B A Management SSP B B Mechanical Engineering SSP B B Universitas Surabaya Chemical Engineering SSP A B Industrial Engineering SSP A A Universitas Syiah Kuala Chemical Engineering SSP C A Chemistry SSP C B Civil Engineering SSP B A Mathematics SSP B A Physics SSP C B Universitas Tadulako Biology NSP NA NA Management SSP B B Mechanical Engineering NSP NA NA Universitas Tanjungpura Civil Engineering SSP B B Electrical Engineering SSP C B Law (International Law) SSP NA B Management SSP C B Universitas Tridinanti Mechanical Engineering SSP C C Universitas Udayana Accounting SSP B B Agricultural Product Technology SSP B B Electrical Engineering SSP C B Management SSP B B Mechanical Engineering SSP C B NA = Not yet Accredited, NSP = new study program, SSP=strengthening study program Note: a. Accreditation A is excellent (score 401 to 500); b. Accreditation B is good (score 301 to 400) c. Accreditation C is fair (score 201 to 300) Source: Directorate General of Higher education 2007

Appendix 8

35

Summary of Loan Utilization

As of 21 April 2008

Uncontracted UndisbursedCategory Description Revised Allocated

Funds

Contracted Balance

Disbursed Loan

Balance 01A Central Project Management

Unit 19,316,093 19,316,093 0 19,316,093 0

01B Sub-Project Management Unit 104,028,095 104,028,095 0 104,028,095 0

02 Front End Fee 1,800,000 0 1,800,000 1,800,000 0

03 Interest & Commitment Charge 23,196,192 0 23,196,192 23,196,192 0

04A Aceh Support Program - Civil Works

1,384,686 1,384,686 0 1,384,686 0

04B Aceh Support Program - Consulting Services

0 0 0 0 0

04C Aceh Support Program - Scholarships

971,437 971,437 0 971,437 0

05 Contingencies 0 0 0 0 0

Special Account 0 0 0 0 0

Total 150,696,503 125,700,311 24,996,192 150,696,503 0 . Source: ADB Loan Financial Information System

36 Appendix 9

SUMMARY OF LOAN DISBURSEMENTS

($ million)

Year Quarter Quarterly Disbursement

Cumulative Disbursement

Percentage of Loan

2001 II 1.800 1.800 1.19 III 1.500 3.300 0.99 IV 0.145 3.445 0.10

2002 I 4.163 7.607 2.76 II 0.242 7.850 0.16 III 0.478 8.328 0.32 IV 13.181 21.509 8.75

2003 I 0.523 22.032 0.35 II 1.164 23.196 0.77 III 1.357 24.553 0.90 IV 5.242 29.795 3.48

2004 I 6.684 36.479 4.44 II 19.012 55.491 12.62 III 11.269 66.760 7.48 IV 9.738 76.498 6.46

2005 I 3.241 79.740 2.15 II 6.208 85.948 4.12 III 5.159 91.107 3.42 IV 16.480 107.587 10.94

2006 I 5.723 113.309 3.80 II 2.853 116.163 1.89 III 2.213 118.376 1.47 IV 13.611 131.987 9.03

2007 I 2.847 134.833 1.89 II 3.973 138.806 2.64 III 2.441 141.247 1.62 IV 8.129 149.376 5.39

2008 I 1.321 150.697 0.88 Total 150.697 100.00 Source: Asian Development Bank Loan Financial Information System

Appendix 10 37

PROJECT IM PLEM ENTATION SCHEDULE

Activities 2000 2001 2002 2003 2004 2005 2006 2007 20081 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1

CPM U ActivitiesAppointm ent of CPM U & PAC

Consulting Services (PM S and Academ ic Consultants)

Im plem entation of Skill Retraining (13.016 Participants)Batch I (1.650 Partic ipants)Batch II (5.433 Partic ipants)Batch III (3.433 Partic ipants)Batch IV (2.500 Partic ipants)

Im plem entation of Student Equity Schem e (20.000 Students)Batch I (3.630 Students, for 4 years )Batch II (3.826 Students, for 4 years )Batch III (5.779 Students, for 3 years )Batch IV (5.473 Students, for 2 years )

Establish and Operate Project M IS/BM S System

Annual Perform ace Review

M id Term Review

Research Study1. Skills Profile , Tracer Study, Public and Private Partnership

2. Review on Com petitive Based Funding, Autonomy, Internal QADissem inate of F inal Report

Independent Technical Audit

Procurem ent Audit

Sub Project ActivitiesBatch I (4 Years)

Batch II (4 Years)

Batch III (3.5 Years)

Additional ISSP3AINew ISS

Additional Com ponentAceh Support P rogram

Yogyakarta Support P rogram

Legend: P lan: Actual

BME = Benefit Monitoring and Evaluation, ISS = Institutional Support Services; MIS = Managem ent Inform ation System ; PAC= Project Advisory Com m ittee; P3AI= Teaching Learning Center

38 Appendix 11

STATUS OF COMPLIANCE WITH MAJOR LOAN COVENANTS

Loan 1792-INO: Technological and Professional Skills Development Sector Project Project Completion Report, 31 December 2007

REFERENCE IN

LOAN AGREEMENT

COVENANT

STATUS OF COMPLIANCE

Schedule 5 Para. 1 Schedule 5 Para. 2 Schedule 5 Para. 3 Schedule 5 Para. 4

1. 2. 3. 4.

Project Coordination and Implementation Project Advisory Committee The Borrower shall, within three (3) months after the Effective Date, establish a Project Advisory Committee (PAC), chaired by the Director General of DGHE. The PAC shall comprise members designated from APTISI, BAPPENAS, MOF, BAN, BHE and other relevant organizations, including private sector representation. The Project Director shall act as the ex-officio secretary of the PAC. The PAC shall review and approved policy reform sin the higher education sub-sector, coordinate and monitor Project activities, review and approved the annual program and budget (for higher education) and assist in resolving inter-agency problems or conflicts arising during Project implementation. Project Executing Agency The Borrower shall cause DGHE, as the Project Executing Agency, to be responsible for the overall supervision, management and monitoring of the Project Central Project Management Unit The Borrower shall cause DGHE to maintain a Central Project management Unit (CPMU) to be responsible for the day-to-day implementation of centrally administered components of the Project, coordination and supervision of Subproject activities and maintenance of up-to-date MIS on the status of Project and Subproject implementation. DGHE shall ensure that in addition to a full-time qualified project Director, the CPMU is adequately staffed with full-time officers for the key components such as MIS/BME, staff development, procurement, the Student Equity Scheme, expert services and studies, administrative affairs, and finance and accounting. Technical Expert Panels The Borrower shall ensure that, as part of Subproject implementation, BHE shall identify and select

Complied with. A Project Advisory Committee was established within 3 months and chaired by DG of DGHE Complied with. The Executing Agency was DGHE Complied with. CPMU was established and well managed Complied with Technical Expert

Appendix 11 39

REFERENCE IN LOAN

AGREEMENT COVENANT

STATUS OF

COMPLIANCE Schedule 5 Para. 5 Schedule 5 Para. 6

5. 6.

Technical Expert Panels (TEP) by 31 January 2001 to assist the CPMU and to facilitate preparation of Subproject proposals and advise on Subproject management. About sixty (60) experts from academic and civil society shall be selected by BHE in accordance with transparent procedures to be approved by the Bank. Subproject Implementation The Borrower shall cause the CPMU to select Subprojects on a competitive basis through the evaluation and ranking of proposals submitted in accordance with published criteria and guidelines to be approved by the Bank. There shall be three (3) rounds of competition during the Project implementation period. Each proposals shall clearly identify the objectives, quantitative targets, monitoring indicators and mechanisms, necessary incremental resources, financing plan, expected economic internal rate of return (EIRR) (to be not less than twelve percent [12%]) and details of counterpart funding from the Subproject Institution (not to be less than eight percent [8%]). The Borrower shall cause CPMU to prepare criteria and guidelines to be approved by the Bank, on the eligibility for and preparation, evaluation, appraisal and implementation of Subproject proposals. The Subprojects shall be selected in accordance with such criteria and guidelines, which shall include the following:

i. Approximately one-third of the Subprojects will be located in west Indonesia; one-third in Java and one-third in east Indonesia;

ii. Public Universities may submit Subproject proposals in any of the five (5) Priority Disciplines of (a) engineering, (b) basic sciences, (c) agriculture, (d) law, and (e) management and accountancy;

iii. Private Universities may submit Subproject proposals in two (2) of the Priority Disciplines of (a) engineering and (b) basic sciences;

iv. Public polytechnics may submit Subproject proposals for (a) engineering, (b) agriculture and (c) commerce including tourism;

v. Private polytechnics may submit Subproject proposals for engineering only;

vi. No Subproject Institution may receive funding under the Project for more than five (5) study programs within an overall ceiling of $8,000,000 equivalent;

vii. A Subproject Institution’s past history

panels were selected by BHE and performed well Complied with CPMU was selected Subprojects on competitive basis; each SPMU contributed more than 8% Complied with Subprojects were selected in accordance with agreed guidelines.

40 Appendix 11

REFERENCE IN LOAN

AGREEMENT COVENANT

STATUS OF

COMPLIANCE Schedule 5 Para. 7 Schedule 5 Para. 8

7. 8.

concerning commitment to operation and maintenance will be closely considered in determining its eligibility to submit a proposal;

viii. Subproject proposals which (a) demonstrate an ability to mobilize local government resources and/or community resources, in addition to the minimum 8% counterpart funding from the Subproject Institution and (b) make innovative proposals to establish or improve community liaison boards shall be given extra points;

ix. Subproject Institutions which have received assistance under current or recently implemented externally-assisted projects will be given a lower priority;

x. A study program must have a BAN accreditation of C or above if the Subproject proposal is to strengthen such study program;

xi. A study program must have a BAN accreditation of B or above if the Subproject proposal is to establish a new, related study program; and

xii. Where a Subproject proposal is to upgrade a polytechnic to a regional polytechnic at least two (2) study programs must be upgraded and strengthened under the proposal.

For the purposes of this paragraph, east Indonesia includes South Kalimantan, East Kalimantan, Sulawesi, Maluku, Irian Jaya, Bali, Nusa Tenggara Barat, Nusa Tenggara Timur and West Indonesia includes Sumatera, West Kalimantan and Central Kalimantan. The Borrower shall cause the CPMU to ensure that submissions for the second round of Subproject proposals shall not be solicited until at least 1 month after the skill profile study, referred to in paragraph 16 of this Schedule 5 has been disseminated in order that Subproject Institutions may incorporate the results of the study in their Subproject proposals, if appropriate. For all Subprojects, the CPMU shall submit the Subproject appraisal reports and draft Subproject Agreements between DGHE and the Subproject Institution to the Bank for approval. The Bank shall conduct a detailed review of at least five (5) Subprojects in each round of competition, selected at random, and all Subprojects whose estimated cost is more than five million dollars ($5,000,000).

Complied with Proposals were submitted on time Complied with ADB approved Subprojects proposed by CPMU after review

Appendix 11 41

REFERENCE IN LOAN

AGREEMENT COVENANT

STATUS OF

COMPLIANCE Schedule 5 Para. 9 Schedule 5 Para. 10 Schedule 5 Para. 11 Schedule 5 Para. 12 Schedule 5 Para. 13

9. 10. 11. 12. 13.

DGHE shall submit a training plan setting out all formal and short-term programs for CPMU and SPMU to ADB for review and approval. Centrally Administered Components Student Equity Scheme The Student Equity Scheme to be established by the Borrower shall assist up to approximately 20,000 students largely from disadvantaged groups. The Borrower shall ensure that the Student Equity Scheme comprises the following features: (i) a scholarship component based on both academic merit and financial need; and (ii) a bridging program of academic tutoring in key subjects and English language for students from less developed regions of the country receiving a scholarship from the Student Equity Fund who require additional short-term assistance to compete at the more developed Subproject Institutions. The Borrower shall ensure that at least 50 percent of the recipients under the Student Equity Scheme are women and that the scheme targets qualified students from families that cannot otherwise afford higher education. The Borrower shall also ensure that lessons learned and suitable individual recipients of assistance under previous Bank projects will be considered for the scheme in order to build upon the benefits gained under the previous projects. Retooling Program Based upon the results of the skills profile and the tracer studies referred to in paragraph 16 of this Schedule 5, the Borrower shall provide re-training programs to assist about twenty thousand (20,000) under-employed and unemployed graduates. Such graduates and trainees to be selected in accordance with criteria and procedures agreed between the Bank and the Borrower. Accreditation Program The Borrower shall ensure that (i) the existing engineering accreditation program developed under the Bank-assisted Engineering Education Development Project will be extended to polytechnics and (ii) an output-based accreditation system shall have been developed and implemented by BAN within 3 years after the Effective Date.

Complied with ADB approved training programs submitted by DGHE Complied with About 20,000 students from disadvantaged families benefited from the scheme Complied with 42% beneficiaries of student equity are women Complied with. In 2005 the target was reduced from 20,000 to 13,000. More than 14,600 graduates benefited from the program Complied with BAN accredited most of polytechnics under the project

42 Appendix 11

REFERENCE IN LOAN

AGREEMENT COVENANT

STATUS OF

COMPLIANCE Schedule 5 Para. 14 Schedule 5 Para. 15 Schedule 5 Para. 16 Schedule 5 Para. 17

14. 15. 16. 17.

Benefit Monitoring and Evaluation The Borrower shall ensure that a Management Information System (MIS) will be developed and implemented by DGHE within 1 year after the Effective Date. The MIS will adequately incorporate the Project BME requirements as agreed with the Bank and in accordance with the Bank’s Handbook for Benefit Monitoring and Evaluation. The Borrower shall ensure that the BME under the Engineering Education Development Project and the Higher Education Project will be adopted to capture the requirements of the Project. Public-Private Sector Partnership DGHE shall carry out a policy study by 31 December 2001 to develop a policy on effective partnership and networking between public and private institutions of higher education. The Borrower shall ensure that the results of the study will be incorporated in the subsequent processing of Subproject proposals. The Borrower shall keep the Bank informed of the results of the policy study and actions taken. Research Studies DGHE shall complete the following research studies by 31 December 2001 and shall ensure that the findings are disseminated to both public and private higher education institutions within 3 months following completion: (i) Skills Profiles for Employability; and (ii) Tracer Study of Graduates in Priority Programs. Additional studies (i) Review on Competitive Based Funding; (ii) University Autonomy; and (iii) Internal Quality Assurance System. Subprojects Participatory Processes DGHE shall cause the CPMU and each SPMU to ensure that the Subproject proposals are designed and to the extend applicable, implemented following participatory practices, as described in the Bank’s handbook Mainstreaming Participatory Development Processes (adapted accordingly).

Complied with MIS and BME established Complied with Policy study on Public-private Partnership completed Complied with Research studies completed Complied with Subproject designed and implemented by participatory processes

Appendix 11 43

REFERENCE IN LOAN

AGREEMENT COVENANT

STATUS OF

COMPLIANCE Schedule 5 Para. 18 Schedule 5 Para. 19 Schedule 5 Para. 20 Schedule 5 Para. 21 Schedule 5 Para. 22

18. 19. 20. 21. 22.

Community and Industrial Linkages Within six months of approval of a Subproject proposal, DGHE shall (i) ensure that the SPMU establishes a community industry liaison board where such a board does not exist; or, (ii) where such a board exist, enhance the activities of the board. DGHE shall disseminate from time to time the best practices in these areas through its web site. DGHE shall ensure that a minimum of 20 Subprojects includes components to establish effective community industry liaison boards. Environment DGHE shall ensure that each Subproject that includes a construction or facility rehabilitation component is designed and implemented, and the Subproject facilities are operated and maintained, in conformity with the environmental impact guidelines of the Government and, to the extent applicable, the Bank’s environmental guidelines. Operation and Maintenance DGHE shall ensure that the Subproject facilities are operated and maintained in accordance with sound technical and management policies and practices; and that the Subproject Institutions provide the required funds, staff, facilities, and other resources to maintain the Subproject facilities. Women Study Centers DGHE shall ensure that at least six (6) Women Study Centers are strengthened under the Project to provide enhanced capabilities and to enable them to participate in the design and implementation of subprojects. Other Matters Financial Matters The Borrower shall ensure that the necessary counterpart funds are made available to the CPMU and the CPMU shall ensure that the necessary funds are made available to the SPMU for the Subprojects

Partly complied Two units of Community and Industrial Relation established Complied with Project facilities operated and maintained in conformity with the environmental impact guidelines Complied with Staff, funds, facilities are provided; however income higher education institutions the integration of a new equipment into curriculum and the maintenance of workshops need to be improved further Complied with Six Women Study Centers established Complied with Government and institutional contributions provided

44 Appendix 11

REFERENCE IN LOAN

AGREEMENT COVENANT

STATUS OF

COMPLIANCE Schedule 5 Para. 23 Schedule 5 Para. 24 Schedule 5 Para. 25 Schedule 5 Para. 26

23. 24. 25. 26.

on a block grant basis and in a timely manner. In this regard, DGHE shall make timely submissions for annual budgetary appropriation requests from the National Budget. The CPMU shall monitor the Subproject Institutions to ensure that the Subproject Institutions shall provide their own counterpart funds as set out in the approved Subproject proposals. The Borrower shall adopt measures to ensure that cost recovery for public higher education institutions increases from the current level of twenty percent (20%) to at least twenty-five percent (25%) by 2005. The Borrower shall ensure that, in order to promote financial autonomy, the Subproject Institutions are permitted to retain and use any self-generated funds as they consider appropriate. Decentralization, Institutional Independence and Governance In the event that authority over financing the higher education sub-sector is devolved to the regions during the Project implementation period, the Borrower shall ensure that the proceeds of the Loan will be allocated and disbursed to the regions, in amounts to be agreed with the Bank, on a timely basis and in accordance with the Borrower’s normal procedures, as developed. The Bank shall, at such time, consider the need to enter into a Project Agreement with participating regions, and if considered necessary, the allocation of funds of any region will be conditional upon the region entering into a Project Agreement with the Bank. The Borrower shall take measures necessary to accelerate the implementation of University Autonomy Regulations (60/1999 and 61/1999) that defines the legal status options of public universities. The Borrower shall prepare an action plan, acceptable to the Bank, for the acceleration of implementation of the regulations by 31 March 2001 and will ensure timely implementation of the action plan as agreed. The Borrower shall keep the Bank informed of the progress in implementation.

in a timely manner Complied with Counterpart funds provided adequately Partly comply with. Cost recovery increased to 25% on average. However, HEI have to transfer self generated revenues to State Account from where it will be transferred on request Complied with Project funds allocated and disbursed in accordance with Project Agreement Complied with. Two project institutions (Universitas Airlangga and Universitas Sumatera Utara) declared as University Autonomy in 2006. The EA failed to submit Action Plan by 31 March 2001; however a plan for autonomy is in place

Appendix 11 45

REFERENCE IN LOAN

AGREEMENT COVENANT

STATUS OF

COMPLIANCE Schedule 5 Para. 27 Schedule 5 Para. 28 Schedule 5 Para. 29 Schedule 5 Para. 30

27. 28. 29. 30.

The Borrower shall ensure BHE implements a time-bound action plan, to be agreed with the Bank, for the acceleration of policy reform and system restructuring and for strengthening of the planning, management and evaluation capabilities at the institutional, regional and national levels in higher education. The CPMU shall maintain and update a web site to disseminate information on the Project, which will contain information, in English and Bahasa Indonesia, about the Project description, objectives, and scope including eligibility criteria and, once available, indicative guidelines for the preparation, appraisal, selection, and implementation of subprojects, and other information such as the results of the technical audit reports. DGHE shall engage independent technical auditors to undertake three annual technical audits of the Project. The audit shall provide an impartial assessment of the Project’s performance including appropriateness of procurement procedures, financial records, budgets and finance, and the selection process of subprojects. The audit reports shall be prepared after each round of subproject award and submitted to the Bank prior to proceeding with the subsequent round of subproject awards. Lessons to be learned from the audit shall be incorporated in subsequent processing of subprojects. The Bank and the Borrower shall jointly carry out a mid-term review of the Project around the fourth quarter of the third year of Project implementation. The mid-term review will review progress in respect of (i) strengthening of sectoral governance and management capacity, (ii) scope and impact of short-term skills retraining, (iii) student equity scheme implementation, and results, (iv) quantitative improvement in student achievement and internal efficiency of Project assisted study programs, (v) student admission in new study programs, (vi) improvement in community or industrial relations, and (vi) sustainability of the Project-forested reforms. Results of the review shall be discussed and appropriate corrective measures will be formulated where necessary.

Complied with BHE performed well Complied with Project information posted in web site by CPMU Complied with Independent Technical Auditor performed well Complied with ADB and the EA executed MTR jointly

46 A

ppendix 12

CONSULTING SERVICES Domestic International

Plan Actual Plan Actual Specialist/Consultant Person Total PM Person Total PM Person Total PM Person Total PM

Centrally Managed Project Management and Implementation Specialist (CPMU) 1 18 2 13.5 0 0 0 0 Quality Assurance Specialist 8 24 2 6.125 1 0 Higher Education Management Specialist 2 24 2 9.25 1 6 3 2.5 Revenue Generating Activities Specialist 2 12 1 0 1 6 0 0 Maintenance management Consultant 2 24 0 0 2 12 0 0 Management Iinformation System Consultant 2 24 1 0 1 12 0 0 Informatics Specialist 1 12 1 0 1 12 Independent Audit Specialists 2 12 4 20 1 6 1 2 Subproject Evaluation and Appraisal Specialists 60 180 87 246 0 0 0 0 Academic Reviewer Specialist 0 0 5 28.75 0 0 0 0 Subproject preparation facilitator 15 30 0 0 0 0 0 0 Academic and Professional Development 0 0 1 2.50 0 0 0 0 Baseline, Midterm, Final Review Consultant 2 12 1 7 1 6 0 0 Student Equity Specialist 0 0 2 6.50 0 0 0 0

Benefit Monitoring Specialist 0 0 6 18.88 0 0 0 0 Subproject Managed

Competency-based Curriculum Consultant 5 30 71 84 5 20 3 3 Engineering Relevancy Specialist 5 20 30 36 0 0 1 1 Laboratory Teaching Methodology Consultant 4 12 36 43 2 12 0 0 Instructional Resource Development Consultant 16 96 56 72 8 96 1 1 Basic Science Curriculum Application Specialist 8 48 29 33 0 0 1 1 Agriculture Curriculum and Instructional Approach Development 4 12 26 28 2 6 1 2 Business/Management Studies 4 12 101 124.25 2 6 5 6 Law 5 20 8 10 2 6 1 1 Accountancy 6 24 17 20 2 6 2 3.25 Computer Science Consultant 12 24 17 20 4 12 0 0 Environmental Engineering Consultant 4 24 0 0 0 0 0 0 Computer System Engineering Consultant 12 24 80 109.50 4 12 1 1 Specialist Training Consultants/Visiting Professors 60 120 22 29 24 48 0 0 Entrepreneurship Training Specialist 12 24 29 32 0 0 2 2 Student Equity Scheme 2 4 0 0 0 0 0 0 Women in Development 3 6 13 13 1 3 0 0

Employer Liaison Consultant 6 12 13 14 4 12 0 0 Total 265 884 663 1026.25 68 299 23 25.75

PM = person-month Source(s): ADB, 2000, Report and Recommendation of the President to the Board of Directors on Proposed Loan to the Republic of Indonesia for the Technological and Professional Skills Development Sector Project ( Loan 1792-INO) , Approved on 29 November 2000, and Reports on Consulting Services Program 2007