Document of The World Bank

FOR OFFICIAL USE ONLY

Report No: 25276

IMPLEMENTATION COMPLETION REPORT(CPL-39790; SCL-3979A; SCPD-3979S)

ON A

LOAN

IN THE AMOUNT OF US$ 60.4 MILLION

TO THE

GOVERNMENT OF INDONESIA

FOR A

SECONDARY SCHOOL TEACHER DEVELOPMENT

June 24, 2003

This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective November 1995)

Currency Unit = Rupiah (Rp) Rp 1 million = US$ 445

US$ 1.00 = Rp 2,246ACADEMIC YEAR

July 1 - June 30

FISCAL YEARJanuary 1 December 31

ABBREVIATIONS AND ACRONYMS

APKG - Student Teaching Observation FormASD - Academic Staff DeploymentCAS - Country Assistance ProgramDO - Development ObjectivesGOI - Government of IndonesiaELA - Entry Level Student AssessmentELAQA - Entry Level Student Assessment in Quality AssuranceIKIPs - Teacher training institutions with education and other faculty departmentsLPTK - General term for Teacher Training InstitutionMGMP - Teacher study circles or clustersMOE - Ministry of EducationPPL - Field Experience ProgramS1 - Undergraduate Degree ProgramS2 - Master's Level Degree ProgramS3 - Docatorate Level Degree ProgramSAR - Staff Appraisal ReportTTL - Task Team LeaderUPBK - Guidance and Counseling Implementation UnitsUT - Open University

Vice President: Jemal-ud-din KassumCountry Manager/Director: Andrew D. Steer

Sector Manager/Director: Emmanuel Y. Jimenez Task Team Leader/Task Manager: H. Dean Nielsen

INDONESIASECONDARY SCHOOL TEACHER DEVELOPMENT

CONTENTS

Page No.1. Project Data 12. Principal Performance Ratings 13. Assessment of Development Objective and Design, and of Quality at Entry 24. Achievement of Objective and Outputs 45. Major Factors Affecting Implementation and Outcome 116. Sustainability 127. Bank and Borrower Performance 138. Lessons Learned 159. Partner Comments 1910. Additional Information 19Annex 1. Key Performance Indicators/Log Frame Matrix 20Annex 2. Project Costs and Financing 24Annex 3. Economic Costs and Benefits 26Annex 4. Bank Inputs 27Annex 5. Ratings for Achievement of Objectives/Outputs of Components 29Annex 6. Ratings of Bank and Borrower Performance 30Annex 7. List of Supporting Documents 31Annex 8. Borrower's Contribuion 32

Project ID: P004003 Project Name: SECONDARY SCHOOL TEACHER DEVELOPMENT

Team Leader: H. Dean Nielsen TL Unit: OEDSTICR Type: Core ICR Report Date: June 26, 2003

1. Project Data

Name: SECONDARY SCHOOL TEACHER DEVELOPMENT

L/C/TF Number: CPL-39790; SCL-3979A; SCPD-3979S

Country/Department: INDONESIA Region: East Asia and Pacific Region

Sector/subsector: Tertiary education (50%); Secondary education (48%); Central government administration (2%)

Theme: Education for all (P); Education for the knowledge economy (P); Gender (S); Social analysis and monitoring (S)

KEY DATESOriginal Revised/Actual

PCD: 10/28/1992 Effective: 05/30/1996Appraisal: 06/08/1995 MTR: 04/01/1999 08/23/1999Approval: 02/20/1996 Closing: 10/01/2001 12/31/2001

Borrower/Implementing Agency: REPUBLIC OF INDONESIA/DIRECTORATE GENERAL OF HIGHER EDUCATION (DIKTI)

Other Partners:

STAFF Current At AppraisalVice President: Jemal-ud-din Kassum Russell J. CheethamCountry Director: Andrew D. Steer Dennis de TraySector Manager: Emmanuel Jimenez Marianne Haug EA3ERTeam Leader at ICR: H. Dean Nielsen Alfonso F. de GuzmanICR Primary Author: H. Dean Nielsen

2. Principal Performance Ratings

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible)

Outcome: S

Sustainability: L

Institutional Development Impact: SU

Bank Performance: S

Borrower Performance: S

QAG (if available) ICRQuality at Entry: S

Project at Risk at Any Time: No

3. Assessment of Development Objective and Design, and of Quality at Entry

3.1 Original Objective:The original project objective was to enhance the teaching-learning processes in secondary schools through the improvement of teacher education in 31 public teacher training institutions. This objective remained thoughout the project, except for the fact that the number of institutions was reduced to 30.

The objective highlights processes in secondary schools, but in reality the project was more focused on the teacher training institutions. Therefore, there was little emphasis, both in project implementation and evaluation, on the secondary school classroom and its processes. In any case, teacher education only covers a subset of the determinants of enhanced teaching-learning processes. Much of the mix comes from the other factors such as presence and quality of teaching-learning materials, teacher incentives, management support for excellence, and community involvement. What the project appears to have focused on -- in both implementation and monitoring -- were the kind of factors that higher educator can leverage -- change within the teacher training institutions and their processes, including those which take place at schools. In retrospect, it may have been better for such activities and processes to have been the focal point of project objectives and for better teaching-learning processes at the schools to have been one of the expected outcomes.

3.2 Revised Objective:No Revision

3.3 Original Components:Component 1: Improving preservice and inservice teacher education ($30.6 million)

This component included curriculum revision (highlighting vertical and horizontal flexibility); provision of science equipment; books and references; offices and conference rooms to select training institutions; student support services and career planning; inservice education; and learning material development.

During a year two supervision mission the Bank team took the position that three of seven original project subcomponents had the potential to directly effect project development objectives (DO) and thus began tracking them in its DO ratings, namely student support services, inservice education, and learning materials development. When the project was scaled back by nearly 50 percent in year three due to the national economic crisis, three of the remaining four subcomponents, science equipment, books and references, and offices and conference rooms were reduced in scope or canceled (offices and conference rooms). The curriculum activity remained, but was so limited in scope that it was also poorly connected to the teaching-learning objective of the project.

Component 2. Strengthening linkages to secondary schools ($5.4 million)

To strengthen linkages to secondary schools the project set out to improve the content of school-based student teaching, redeploy substantial numbers of teacher training institution (LPTK) lecturers (who at the beginning of the project were in oversupply) to secondary school teaching positions, and promote collaboration between teachers and lecturers in conducting school based action research.

All aspects of this component can be considered directly and strongly related to school teaching-learning processes and were appropriately used in assessing attainment of project objectives. The international partnerships formed as a way of enhancing the project resource base were well conceived and useful at the beginning but eventually domestic expertise appeared sufficient to drive these activities and given the need

- 2 -

to scale back on budget, the funding for international partnerships was appropriately cut.

Component 3. Raising qualifications of teacher educators ($32.4 million)

This component, meant to strengthen the professional qualifications and expertise of LPTK lecturers, consisted of fellowships for national and international degree programs (Masters and Doctoral), refresher courses at international centers of excellence, and related national workshops.

Given the fact that at Project appraisal a high proportion of teacher educators (80 percent) had only received a bachelors degree (the same degree that they were offering new secondary school teachers) -- and most from the institution where they were teaching -- this component can be considered an essential means of raising professional competence related to secondary school instruction. When project recosting was done, many of the subcomponents were reduced (especially the one covering the less efficient refresher courses), but even at their scaled-back levels all subcomponents were maintained as proxies of development objective attainment.

Component 4. Building educational research capacity ($2.9 million)

This component consisted of developing and using a system of entry level student and graduating student assessment, and competetive research grants to encourage instructional improvement.

Given the fact that a large proportion of students admitted to the teacher training institutions have not mastered the subject matter that they will be expected to teach (e.g., secondary school math or science or language skills) a diagnotic tool and its use to guide remedial actions (tutoring) can be considered an important potential contributor to project objective attainment. Use of the instrument for quality assurance (in this Project meaning to test both entry-level and exit level mastery) also has the potential to leverage program improvement. Also, given the low priority given in the teacher training institutions to effective instruction by the lecturers themselves, the competitive grants were welcomed as highly visible means of changing that. It is reasonable to note these subcomponents were included among the nine proxies of project objective attainment.

Component 5. Preparing for a wider mandate ($1.1 million)

This component set out to examine, in four teacher training institutions (IKIPs) making the transition to university status, issues of strategic planning, accreditation, and student career planning.

This component was never well connected to the project objective of fostering better teaching and learning in secondary schools except in a negative sense -- converted training institutions were at risk of losing their motivation and capacity for excellence in teacher education. Given that even before the Project results on this component were ready, all ten public IKIPs converted to universities, the importance of this component in policy formulation and program development was undermined. It was appropriately not included as a proxy for project objective attainment.

3.4 Revised Components:All of the components were maintained throughout the project, but their levels of funding were changed due to the cost-cutting requirements, as follows (the letter after the cost figure reflects their ratings at the time of cost reduction): 1. Improving preservice and inservice teacher education ($9.5 million); S2. Strengthening linkages to secondary schools ($3.8 million); S

- 3 -

3. Raising qualifications of teacher educators ($19 million); S4. Building educational research capacity ($3 million); S5. Preparing for a wider mandate ($0.8 million); NR

3.5 Quality at Entry:Although no formal Quality at Entry assessment was made of the project, it is possible to get a sense of the adequacy of project preparation and design through document review, namely by project adherence to the Country Assistance Strategy and by its response to external reviews. The 1995 Country Assistance Strategy for Indonesia called for a lending program that would address critical issues in human resources development, especially those related to educational quality, and for the adoption of strategies supportive of the country's transition towards more regional autonomy, including that in higher education. The project was well designed to deliver higher levels of educational quality at the teacher training institutions, and placed appropriate and adequate emphasis on institutional development and autonomy. Less well designed were features that would allow the development objective of improving the teaching-learning processes in secondary schools.

Project external reviewers were supportive of the main features of institutional improvement for teacher training but questioned whether the project had built adequate mechanisms for coordination and communication among institutions, such that innovations and changes in lead institutions would be shared across the entire network. Related to that were concerns about the lack of engagement of accreditation or national policy bodies in the exercise of quality assurance. In fact, there were problems of communications and sharing and quality assurance was not a strong project feature. A national Directorate for Teacher Education was established during the life of the project but too late for it to have a major impact on policy change. Another point by reviewers was the project's lack of attention to changes in the labor market for secondary school graduates and issues of national competitiveness in productivity. Although it could be asserted that curriculum design for secondary schools was beyond the scope of this teacher development project, it can be argued that the project should have paid more attention to both the content and processes of teaching and learning in secondary schools. Finally, there was a reviewers' call for a more coherent and forward looking view of the teacher education system and its organization, given the pending transformation of training colleges into universities. In retrospect, it seems that system was overwhelmed by the scope and pace of transformations that did take place, which might have been better absorbed had more strategic planning and holistic thinking been done.

4. Achievement of Objective and Outputs

4.1 Outcome/achievement of objective:The attainment of the project object, "to enhance the teaching-learning processes in secondary schools through the improvement of teacher education in 31 public teacher education institutions," can not be directly assessed, at least the first part, since the project did not systematically collect data on changes in teaching and learning at the secondary school classroom level. What can be assessed is the improvement of teacher education, as follows:

Improving pre-service and in-service teacher education Curriculum. A key task in this component was the preparation of new curriculum materials in English, social sciences, and subject-specific pedagogy and the creation of vertical, horizontal, and external flexibility. Though the curricula have been field-tested, and, in the case of intensive English course materials, adopted in many institutions, system-wide adoption was in doubt in late 2002 (near the end of the project) since national guidelines released at that time leave it up to each institution to create its own

- 4 -

curricular materials. Thus, the Project materials were distributed to the group of 31 institutions as exemplary curriculum materials which the institutions could use in creating their own. Vertical flexibility (the ability to teach at both lower and upper secondary school levels) was improved through specific changes in social science and subject specific pedagogy courses and field experience programs. External flexibility was found to some extent in the English language programs. However, horizontal flexibility (endorsement to teach more than one subject matter) was not established during during the Project. The main vehicles to be provided, the use of double majors or major-minor options, were not observable during the project, since such options were only seen as coming at the end of a program and the first cohort of those experiencing Project revised curricula had not yet finished their initial majors.

Science and language laboratory equipment. Science equipment provided under the project, while utilized in some laboratories, was not used in other places due to a lack of complimentary material, e.g. software, nitrogen gas, etc. Problems in the maintenance of science equipment were also noted, at least in part a problem of inadequate training. Foreign language laboratories, used mostly for English language instruction, were well received and intensively used, but the lack of student keyboards prevents students from being able to use the labs to improve their writing skills.

Distance education modules. The lack of enthusiasm for the Open University (UT) approach to inservice teacher training activities (upgrading), and its eventual discontinuation under the project, has meant that some 120 modules developed by the Open University for teacher upgrading (20 more than the target) are not being utilized widely in the campus-based teacher training programs. However, they are being used as basic texts for the distance inservice training programs managed by the Open University (ten of thousand of students) no longer included as part of this Project. If the best of these modules could be made available to the other teacher training institutions for their inservice programs, the returns on investment could be increased.

In-service education. A major potential contribution has been made to the improvement of classroom instruction by the Project's placing 18,725 lower and upper secondary school teachers in inservice training programs for their upgrading to the S1 degree. Both faculty and participants spoke positively about the program which was selective and rigorous. Of those placed almost 15,000 graduated (an 80 percent completion rate); of the graduates 56 percent were women, and 58 percent lower secondary school teachers (compared to the originally planned 77 percent). It is not clear why lower than expected numbers of lower secondary school teachers received the scholarships. With the cancellation of the Open University program, the project was less well equipped to reach rural and remote school teachers than planned -- although some LPTKs provided remote access to courses and outreach was still being made by the Open University through its own resources. Studies showed that inservice course participants did just as well as inservice program graduates on cognitive outcomes and teaching skills (based on head teacher and student feedback). (Unfortunately, the study did not include the kind of direct observation of teaching skills used in the regular preservice courses.) The program was undeniably popular among teachers, evidenced by its continuation with those willing to pay their own fees and the demand for it by private universities and distant provinces, presumably because of its help to them in reaching higher levels in the civil service (an important incentive for a serverly underpaid and under-appreciated teaching force). It also showed its capacity to increase teacher knowledge -- important in a teaching force where even mastery of the secondary school curriculum is weak. Improvement in teaching skills was probably not (and probably cannot be) strongly affected by such programs, since generally this requires more "hands on" approaches, like those featured in the action research programs and faculty deployment systems described below under university-school linkages.

Student support services. Student support services (3S) have been established in 30 LPTKs to improve student study skills and adjustment to university life, but their best benefits are only being realized in about

- 5 -

a third of them, the ten which have developed entry level tests to identify weak subject mastery and are providing tutoring for the underprepared students. Even in those the most needful students (those with low entry level scores) are not always reached since tutoring is not required. The Centers are managed by staff trained in guidance and counseling, not academic remediation. Some remediation activity seems to be done at subject departments, e.g. English at UNM and not at the center. One notable feature, used in many locations, but whose effectiveness is not yet established, is peer tutoring. While well-qualified students can help their peers, other limitations, e.g. time, teaching ability, may not be adequate. Career counseling has also been established at some universities, but no data available to assess how effective it is.

Improved linkages to secondary schoolsThere was a significant increase in the activities between the teacher education institutions (LPTKs) and the schools through highly valued classroom action research and deployment of 902 lecturers to schools. The classroom action research which partners LPTK lecturers with classroom teachers in examining the teaching learning process in schools increased lecturer knowledge of current practices in the secondary schools and connected them to the real world of teachers. The deployed lecturers worked as teachers, on action research, and community service. A model field experience program was developed that included a handbook and manual that was tried out with partner schools at all LPTKs. Supervising lecturers and cooperating teachers were trained in the use of the field experience handbook and manual. There was no systematic pre-/posttest data collected to determine if the performance of student teachers changed as a result of the increased linkages to secondary schools; nor do we know if lecturers/teachers improved their teaching as a result of the school linkages program and their action research. Observations during site visits and feedback from lecturers, teachers and administrators were supportive of the linkages and some LPTKs are using their own budgets to continue these activities beyond the project. Partnerships with international universities were supported for the first two years of the project. The economic crisis curtailed project support but five LPTKs continued the partnerships on their own.

Raising the qualifications of teacher educators Graduate degrees. Using domestic universities and creative means of accessing international ones (particularly in Germany, France, and Holland, where university placement agents were used) the project delivered most of its planned graduate degrees. Virtually all masters and most doctoral programs were in the subject fields or disciplines, which was the plan at the masters level but not the doctoral where a balance of disciplinary and pedagogical studies was expected. Also, the process of candidate selection and placement appeared less reliant on institutional needs analysis than expected. The drop-out rate from degree programs has been quite low (less than 10% for the international doctoral candidates) but several doctoral programs are still in progress (about 25%). For any students needing time beyond the project closing date to finish, the universities (or in a few cases the students themselves) have agreed to cover the remaining costs. Masters and doctoral returnees have almost all returned to teaching in their former departments, and show increased levels of academic output (research, journal writing, involvement in academic conferences). There is some evidence (self-reports and comments from Deans) of improvement in their own teaching and reports of higher satisfaction among students about the returning lecturers' instruction and thesis supervision (based on tracer studies of 50-60% of returnees) -- but there was no systematic/objective assessment of improved teaching competence. Few of them gained specific training and new expertise on improving quality of teacher education programs; most are more oriented to the non-education programs of the wider mandate institutions and often research that can only be done overseas (e.g., physics research in France). The overall goal of the project – to increase the proportion of faculty holding a post bachelor’s degree to above 45 percent -- was exceded (as of 2001 it was 51.5 percent; counting faculty in the pipeline 63.2 percent), with as much as one third of the change attributable to the project.

- 6 -

Refresher courses and faculty workshops. Of the 150 refresher courses that were planned only 22 – the first round -- of them were undertaken. Since they were found to be relatively expensive (compared to potential impact), subsequent rounds were canceled during the cost reduction. Of the 22 only eight yielded booklets which were published and distributed. There is little sign of their having been used (except by producers and reviewers) or visible/available at the receiving institutions – and no faculty workshops on them were held, as planned.

- 7 -

Building Educational Research CapacityEntry level student assessment (ELA) and Quality Assurance. ELA instruments were developed at ten LPTKs and have been used constructively by the institutions to identify students lacking pre-requisite knowledge (secondary school content) and prepare remedial programs, usually at the student support service centers. Some institutions have made remediation mandatory and some not; also some use only faculty members as tutors and whereas others involved peer tutors. Feedback directly to faculties and departments for use in course development does not seem to have been done systematically. The use of the instrument for quality assurance (end-of-program testing) was done, but at most for two years. In some programs there was significant change; in others not, a finding at least in part influenced by the fact that students and faculty did not take the post-test very seriously. At least one institution showed that results were better when student tutoring was undertaken. The intended use of the student teaching observation form (APKG) for summative evaluation of trainees' teaching skills was done only at two institutions with results which are not conclusive.

Research on the improvement of instruction. Originally meant to cover up to five studies per year, this program grew to significant proportions (229 grants awarded in all) and was accompanied by publications and workshops which have had a positive impact on lecturer attention to their own teaching and student outcomes. However, there are no data to actually document improved instruction.

Preparing for a Wider MandateOriginally the project proposed the preparation of two to three IKIPs for future conversion to universities. Four IKIPs were selected to participate in conversion to university status with support from the project. Institutions preparing for conversion were to provide graduates with three types of competencies: vertical - the ability to teach at lower and upper secondary levels; horizontal - the ability to teach 2 to 3 subjects; external - the ability to compete for employment other than teaching. The time needed to grow and develop into a university was estimated at between 5 and 15 years. As IKIPs came to recognize the status improvements connected to this conversion they all applied for and were granted the permission to convert during the project period. This process was accelerated by a Presidential Decree in support of the conversion in 1999. The impact of these conversions on teacher education has yet to be assessed. One of the immediate results is that the academic qualifications of the intake students has increased, nearing and even surpassing that of the established universities. A stimulating academic environment may positively impact teacher education if the better qualified students select teaching as a career. However, there is no guarantee that these students will pursue teacher education. As non-education programs increase, competition for resources could adversely affect teacher education. Thus, during the project the goal of converting a few IKIPs to universities was "overfulfilled" but with questionable results with respect to the quality of teacher education offered and teachers produced. It is crucial that former IKIPs monitor this during the first years of their new wider mandate status, tracing number and characteristics of their graduates who enter the teaching force, and making changes in their programs if it turns out that too few of their good students are going into teaching.

The accreditation system, including peer evaluation elements, proposed in the project evolved into a less rigorous self-evaluation system. Under the direction of the Director General of Higher Education, self-evaluation reports are now required as part of the institution’s annual budget request, a positive first step. However, stronger accreditation measures will be needed if the government is serious about improving the quality of teacher education, especially given the increased number of private universities and faculties offering teacher education.

4.2 Outputs by components:Improving pre-service and in-service teacher education

- 8 -

Curriculum improvement. Three curricula developed and pretested: Intensive English in 8 LPTKs, Social Sciences in 3 LPTKs, and Subject-Specific Pedagogy in 3 LPTKs. The horizontal flexibility features of the English and Social Sciences curricula (allowing double majors and major-minor) were not tried out during the Project implementation.

Science equipment. Using the Basic Science Team Standards, biology, chemistry, mathematics and physics equipment was purchased and installed in 12 of 16 planned LPTKs. Support for the last 4 was taken out of the Project under cost reduction and covered by GOI funds. At one institution the science laboratory was renovated and is used for S1 students practicum. Science laboratory storage facilities were installed at two LPTKs. As an added feature to the Project, English language laboratory equipment to support the implementation of the English curriculum developed by the SSTD English Development Team was purchased and installed in 6 LPTKs; in two others supplementary language laboratory equipment was provided.

Secondary textbooks and education references. Textbooks used in lower and upper secondary schools were purchased and distributed to all LPTKs. Biology, chemistry, and physics textbooks (7 of the targeted 75, scaled back to 12 under cost reduction) were translated from English to Indonesian, published, and distributed to all LPTKs except UNPATTI due to civil disturbances in Maluku. Eight of the 22 refresher course program textbooks (scaled back from 150) were published and distributed to all public LPTKs.

Student support services. Student Support Centers were established at 30 LPTKs. A model program was set up at the State University of Padang (formerly IKIP Padang) and National Seminars held to discuss the "3S" program and its effectiveness as documented in studies at 14 LPTKs. Students having low Entry Level Scores were identified in 7 LPTKs and other instruments for detecting learning problems were developed and used in 30 LPTKs.

In-service education for secondary school teachers. A total of 18725 scholarships for upgrading secondary teachers from D3 to S1 were awarded, exceeding the project target of 18,000 (14,000 for SLTP and 4,000 for SMU). Upgrading of 600 teachers through the Open University was cancelled due to cost containment.

Learning materials development. Basic course outlines were developed and reviewed for twelve subject-specific pedagogy areas. Modules (120 compared to the planned 100) were produced for use in the distance learning program of the Open University (UT). Since upgrading via UT was not undertaken under the project, the modules were not used in the project. However, they are still being used by UT for with its non-project supported students.

Offices and conference rooms. Staff offices were remodeled and renovated at 12 LPTKs and are being designated for meetings between students and lecturers as evidenced by office hours posted on office doors. However, there are not data to show whether such office hours are being maintained by lecturers.

Strengthening linkages to secondary schoolsSupporting school linkages. A total of 912 lecturers, from eleven LPTKs, were redeployed to secondary schools under the academic staff deployment (ASD) program (well below the 2400 mentioned in the SAR but reduced to an undetermined number during cost-reduction). They worked in a variety of activities: teaching, action research, student counseling, learning material improvement, and community service. For Field Experience Program (PPL) a handbook and manual were developed. Tryout of new PPL models were conducted by LPTKs and partner schools in all LPTK. There was a total of 180 partner schools with an

- 9 -

average of 6 partner schools per LPTK. A total of 6140 cooperating teachers and supervising lecturers were trained to use PPL materials (far above original expectations). At total of 559 classroom action research studies (far more than planned expectations), conducted by teacher-lecturer partners, were selected for funding in five batches. National seminars were held where findings from research were reported. National evaluation of CAR was conducted and results shared at a national seminar in December 2001.

Partnerships with foreign institutions. IKIPs were partnered with international universities with strong teacher education programs. Ohio State University was partnered with Malang, Bandung, and Surabaya; the University of Iowa with Padang, Medan, and Manado; Curtin University of Technology with Semarang, Yogyakarta, and Gorontalo; and LaTrobe University with Makassar, Singaraja, and Jakarata. International visits followed by in-country workshops for dissemination of new ideas were conducted during the first two years of the project before the international activities were discontinued as a result of financial crisis. Some LPTKs (Makassar, Malang, Surabaya, IKIP Singaraja, Bandung) continued the international partnerships with their respective partners (OSU and La Trobe) even after project funding was terminated.

Raising the qualifications of teacher educatorsMaster’s programs. Domestic S2 program fellowships and international masters programs were awarded to 1301 and 57 teacher educators, respectively. The original targets were 1450 in national universities and 150 in international universities, adjusted to 1250 and 60, respectively. The completion rate for the domestic masters program was 84 percent. The international masters degrees were taken in Australia (28), United States (4), Canada (12), United Kingdom (12) and New Zealand (1). The completion rate for the international masters program was 93 percent.

Doctorate program. Domestic doctoral program fellowships taken in academic departments in prominent universities were awarded to 16 teacher educators with appropriate S2 qualifications (compared to 20 planned -- adjusted from 50). International doctoral fellowships were awarded to 105 teacher educators (the original target was 100). About half (53) of the doctoral students are still completing their programs; thus this part of the project is being extended to December 31, 2002. Countries for doctoral studies are: Australia (17), United States (4), United Kingdom (2), Holland (27), France (33), Germany (21), and Malaysia (1).

Refresher courses and faculty workshops. Short-term refresher courses were made available to 22 teacher educators (of the original 150 planned -- reduced due to cost-cutting). Course-related materials were developed by the returnees from the refresher program. Of the materials developed 14 were deemed worthy of publication. These were revised, eight were published and distributed to all the public LPTKs.

Building educational research capacityEntry-level student assessment. Diagnostic tests to assess entry level knowledge (ELA) for 10 study programs (mathematics, physics, biology, chemistry, English, B. Indonesia, history, geography, civics, and economics) were developed and administered at 10 LPTKs (no clear target number). Results of ELA were disseminated to 3S teams at LPTKs. Program coordinators and lecturers from the other 20 LPTK attended ELA training conducted in Bogor and Surabaya in Feb. 2000.

Quality assurance. Training for use of ELA in quality assurance (ELAQA) in four subjects was conducted at 30 LPTKs. Exit-level testing of content knowledge was conducted at 4 LPTKs (UM, UNM, UNESA, and UNP). Exit level assessment of teaching skills was conducted at two institutions, UNESA and UNP. (No target numbers were specified).

- 10 -

Research on the improvement of instruction. Due to high demand and large number of proposals this program was expanded from an original target of 5 studies per year to over 50 per year (229 in all over four years). Each year (from 1999-2001) national seminars were held to discuss the studies' findings. The results of twenty of research studies were published in Parameter, an accredited education journal of State University of Jakarta.

Preparing for a wider mandatePlanning for a wider mandate. All ten public IKIPs became universities (compared to the planned two to three) and the two public STKIPs to IKIPs (unplanned).

Preparing for accreditation. A guide book for self-evaluation was completed and distributed for required use by all LPTKs.

Career planning. Career information services were set up at UPBK (Guidance and Counseling Implementation Units) in 12 LPTK. Training for the other 18 institutions conducted in 1999. Information was developed regarding types of competencies, occupations, vacancies, and training in education and non-education careers relevant to LPTK graduates.

4.3 Net Present Value/Economic rate of return:NA

4.4 Financial rate of return:NA

4.5 Institutional development impact:During project implementation the Government of Indonesia created the post of Director for Teacher Education within the Directorate General for Higher Education, an important institutional development which had been recommended by the Bank since the launching of the Primary School Teacher Development Project in 1992. This appointment, although coming later than hoped, has provided teacher educators a voice in the Directorate General and created a locus for policy and program development. It came at a time of Indonesia’s move to more university autonomy and accountability, and after the conversion of ten public teacher training institutions to universities (IKIPs). That conversion was another important development impact of the project. Expected to proceed slowly, using planning and institutional development instruments created under the project in two to three institutions, the conversion quickly gathered momentum and by the end of the project had occurred, for better or for worse, in all 10 public IKIPs. Under the project’s component “preparing for a wider mandate,” the first step in an accreditation processes (institutional self-assessment) was launched which the Director General now requires to be used and submitted along with budget requests annually. One of the observable implications of the conversion is an increase in the quality of applicants to the former IKIPs, in some places now at par with that of more established universities.

At the project level, a high level of managerial capacity and productivity was noted in Bank supervision documents. All project components were managed by prominent teacher educators, many of whom are at the forefront of institutional reform in their own institutions. The team, which grew in effectiveness over the years, was remarkably stable throughout project implementation (especially given the fact that the nation’s President and Minister of Education changed four times). Some external technical assistance was sought in relation to graduate student placement and other technical features of the project, but for the organization and production of project outputs all management was done by Indonesia’s own teacher

- 11 -

educators.

At the university level, institutional development is often portrayed in terms of the professional qualifications of the teaching staff. Here there were also dramatic developments during the project (although not only as a result of the project). Whereas at the beginning of the project over three quarters of lecturers held the S1 degree (generally from the institution itself), at the end more than half were possessors of the S2 or S3 degree; counting those lecturers currently in S2/S3 training the proportion stands at 63 percent. Project managers and university officers comment on improved academic environment that has resulted from this shift. In addition, many project innovations are being institutionalized. For example, many institutions now routinely do entry-level assessments of new students and provide remediation for those whose prerequisite knowledge is deficient. Remediation is being conducted under the Student Support Services which have a permanent place on campus together with Career Planning Services, including those for students who are non-education majors. Also expected to be covered by routine budgets at the participating institutions are the improved methods of student teaching (codified in implementation manuals), lecturer redeployment to secondary schools (temporary attachments as teachers and partners in action research), and classroom action research, often funded the LPTK's research center.

5. Major Factors Affecting Implementation and Outcome

5.1 Factors outside the control of government or implementing agency:Near the middle of the projects second year (late 1997) a financial crisis swept through Asia, and hit Indonesian particularly hard -- in 1998 there was a negative growth rate of around 15 percent. Foreign investors and capital fled the country and severe devaluation (and attendant inflation) ensued. This put severe pressure on the national budget and foreign exchange reserves leading to a policy decision to reduce indebtedness. The direct consequence for the Project was almost a 50 percent reduction in the size of the World Bank loan. The major indirect influence was the impact on schools (and thus the attainment of quality improvement goals): by 1998-99 schools began experiencing real declines in financial support (covering books, facilities, etc.); teacher pay increases were undercut by inflation. In Indonesia the financial crisis was accompanied by political crisis – the resignation of President Suharto after more than thirty years of absolute rule, followed by three new presidents in the course of two years. With the four changes in presidency came four changes in the Minister of Education, which meant constant changes in MOE direction and emphasis and uncertainty for Project strategy.

5.2 Factors generally subject to government control:By the end of the Project’s second year government budgets for education had diminished in real terms, including that for teacher salaries, instructional materials and school maintenance. Thus, even if the project was doing a better job of preparing teachers and improving learning processes, other contextual factors mitigated against quality improvement. In addition, by 1999 the Government promulgated an ambitious policy of decentralization (to the district level), which has not yet been well articulated in education service delivery and has led to much confusion as to who is in charge of what. By the end of the project, when decentralization was in full swing, there was concern about district's ability to maintain even the current low outlays for education – factors further threatening school quality. A policy related to teacher education, already in place in the project's first year, was that of zero growth in government staff positions. The fact that a low proportion of LPTK graduates were being hired acted as a damper on interest in the teaching profession and drove many potential teachers into non-teaching fields. Another major personnel issue is the fact that full time university lecturers are appointed as civil servants having tenure from the beginning. This prevents the university from using some of the conventional performance indicators -- excellence in teaching, research productivity, and university/community service -- as incentives for promotion and tenure. Instead university staff are drawn into project work, consultancies, and outside

- 12 -

teaching as ways to increase their stature and incomes.

5.3 Factors generally subject to implementing agency control:The project got underway quickly with a management team more skilled in moving materials and infrastructural development than quality improvement, resulting in some delays/distortions in the latter. Also, the Director General in place at the beginning of the project, using the prevailing ideology of providing the same assets to all institutions, overrode project design features of selective strengthening (using the project to strengthen lead institutions which would influence the others). Among other things, this meant that more science equipment was distributed than was called for in the plan, equipment which later was found to be underutilized and poorly maintained. In addition, the Directorate General relaxed controls over the number of teacher training institutes that could be converted to universities and the processes for making the transition. As a result all IKIPs made the transition with unknown consequences with respect to the status of teacher education. In the project management there was a reluctance to recruit consultants, for reasons which are unclear at the beginning but eventually related to cost reduction efforts. As a consequence none were used. This seems to have had two opposing impacts: one was positive pressure on project managers and team leaders to develop the needed management capacity (see section 4.5 on institutional development impact); the other was a missed opportunity for quality improvement inputs, especially on project features such as curricular design and quality assurance systems.

Certain MOE policies also had an impact on project implementation. First the policy to create a new teacher education curriculum by 2002 (the first post-project year), using a competency-based model. Since new curricular guidelines were only expected to specify required outcomes and leave the ways of reaching them up to the institutions (under principles of institutional autonomy), the kind of national curriculum that was being prepared by the project was out of date by the project’s end. Second, the Ministry made a decision to allow LPTKs to conduct inservice teacher training. Previously, this had only been the realm of the Directorate General of Primary and Secondary Education assisted by the Open University. The decision to allow other institutions of higher education to do this brought large numbers of new clients to LPTKs, allowing them to be selective. That plus the wider mandate helped former IKIPs to increase their student teacher ratios, on average, from 9.4:1 to 13.4:1, resulting in large improvements in institutional efficiency.

5.4 Costs and financing:A significant cost reduction effort was undertaken during the project’s third year at the request of the Ministry of Finance, which was under pressure to reduce indebtedness. For PGSM this meant reducing Bank financing from a total of US$60.4 million to US$30.4 million. It was extremely difficult to cut that much from the budget, and some components clearly suffered (see above). Nevertheless, the project adjusted and eventually accommodated to the lower funding levels. In fact, in the end the Project had a balance of just under US$2 million, most of which was from the category of international fellowships (19 percent of the total allocated for that category). Some of these programs were less expensive than expected, some were covered by other sources of funds, and some were not completed before the end of the project (also requiring other sources of funds for completion).

6. Sustainability

6.1 Rationale for sustainability rating:The Government has put particular emphasis on sustainability during the project’s last two years. Planning for it has included creating field-tested manuals covering such things as student teaching and career counseling, holding dissemination workshops, creating new incentives (e.g., for action research) or exploiting olds ones (e.g., for inservice training), and pressing for funding commitments from both the central government and the teacher training institution. A promising development in the current, first

- 13 -

post-project year has been the creation of a government funded project which will continue the work of transition planning and institutionalization during the next four years (see below). Institutionalization is already apparent in many LPTKs where project programs are being covered by the institutions' own budgets. Given the reduction of funds for higher education during the prolonged economic crisis in Indonesia, financial support, even for the best programs, will be hard won.

6.2 Transition arrangement to regular operations:6.2. Transition arrangements to regular operations:As mentioned in 6.1 above, the Directorate General has mounted a government-funded project which is managing the transition to regular operations: the institutionalization of project innovations. Many of the PGSM managers are members of this transition. The team will evaluate project-developed programs and products and make recommendations about what to institutionalize and how. Transition managers are distinguishing between further development costs, which the center might fund, and recurrent costs, which should be borne by the teacher training institutions.

There are indications that individual institutions are beginning to cover project costs with their own routine funds. For example, some institutions will put funding in their research center budgets for the kind of classroom action research pioneered by the project. Similarly, some institutions are already bearing the costs of entry-level assessment, tutoring for under-prepared students, and expanded/improved student teaching. Academic staff deployment (temporary placements of lecturers in secondary schools) is popular at many institutions but funding my be hard to obtain. In at least one institution (UN Malang) arrangements were made for the head of the local educational office to take a study tour to the US, funded by the university’s international partner. This educational office head now provides local government funding support for staff deployment and other LPTK-school linkages. Transition team members are also trying to convince universities to use non-monetary incentives for implementing project innovations: for example, helping staff to get the maximum functional credit units (used for promotion) out of the activities undertaken cooperatively under school linkages (training, research, and teaching).

Finally, some project activities will continue because of incentives that are already in place. For example, secondary school teachers will continue to seek inservice training, and pay for it with their own money, because it contributes heavily to their own promotion. In fact, it is likely that project funding never was needed to get this moving; only the permission granted by the Ministry of Education to LPTKs to conduct inservice training (originally only given to the Open University). Such training could be further extended beyond the regular catchment of the university by granting the LPTKs permission to do distance education. Raising the qualifications of teaching staff will also continue since the motivation of staff is high and alternative sources of funding keep appearing, both foreign (bilateral agencies) and domestic.

7. Bank and Borrower Performance

Bank7.1 Lending:The project was designed to support the Country Assistance Strategy (CAS) for Indonesia that was presented to the Board on March 21, 1995. The Bank’s strategy called for continued assistance to Indonesia in meeting its development objectives through a lending program that would address critical issues in human resource development, especially those related to education quality: increasing expenditures on education, better distribution of teachers, staff training, and learning materials. The focus of assistance in education and training was to be decentralized to the regional and provincial levels with emphasis on the individual teacher training institution as the unit of development – in case of this project to the 31 LPTKs. Project preparation included a small group of professionals with a good mix of experience in both teacher training institutions and Indonesian schools. One of those professionals continued as a

- 14 -

consultant to supervision missions during the entire lifetime of the project.

A major shift in Project strategy came in mid-1998 when, given a major financial and budget crisis, the government asked for a major cut in the Project budget. The Task Team Leader was asked to reduce Bank financing by almost 50 percent. The TTL, with the help of economist from the Bank, constructively engaged the Directorate General and project management in adjusting output targets and unit costs to reflect current prices and feasible implementation. Costly activities, especially those involving expenditures in foreign currency, were scaled down or eliminated. Project activities considered essential to achieving Project development objective were jointly identified by project management and the Bank mission and recalculated for loan financing, including student support services, in-service education, books and references, linkages with secondary schools, deployment of lecturers to secondary schools, international partnerships, entry-level assessment, and research on instruction.

Restructuring resulted in the cancellation of the third batch of science equipment, refresher courses, upgrading through UT, and remainder of office renovations. Targets for national and international masters degrees were adjusted as were national targets for doctoral studies.

One unexplainable lapse was lack of any performance indicators for the main development objective of the project, the enhancement of the teaching-learning processes in secondary schools. Proxies for this were developed and used in the creation of supervision ratings, but the question still remains: did these processes improve? At the very least there could have been a study of the improvement of classroom teaching skills among the 15,000 teachers who received inservice training.

7.2 Supervision:Bank supervision during the life of the project was remarkably stable with the same task team leader serving for four of the Project’s five years and his replacement being a person with over 20 years of experience in Indonesia, including that in teacher education. Also, the same teacher training consultant joined every supervision mission. Nevertheless there was some momentum lost in the transition from one TTL to the other, especially in the tracking of project impact indicators and in planning for sustainability. In 1997 a rating scale was introduced for use during supervision missions. This rating scale was taken seriously by the project management team as it represented an average of those components contributing directly to the Project’s development objective. With the introduction of the rating scale, project productivity increased and the overall project rating was considered to be more transparent.

Mission reports show some gaps in the Bank’s financial management and supervision. Reports to the Government on disbursement were seen as irregular and supervision of procurement somewhat problematic. The decentralization of higher education moved procurement to the various teacher training institutions with the CPIU acting as reviewer. Insufficient Bank-led training at these institutions in procurement and financial management meant that Bank guidelines were often not followed nor standard formats used. However, post-review of documents revealed no substantial procurement irregularities during the life of the Project.

7.3 Overall Bank performance:The overall rating of Bank performance is satisfactory.

Borrower7.4 Preparation:At the preparation stage, a six person team was designated to design the project. The work of this team was fully supported by the Directorate General of Higher Education and WB Consultants. International

- 15 -

study visits were made by team members to teacher education institutions with particular emphasis on institutions having strong teacher education programs and those converted from normal schools to colleges or universities. During preparation the Borrower took seriously the Bank’s concern for institutional efficiency and agreed to a redeployment of some LPTK staff to secondary schools in order to raise the average student-faculty ratio to 12:1. This was also consistent with the Government’s personnel policy at the time of zero growth. A good beginning, and a practice repeated during the project was the bringing together of LPTKs and stakeholders in Secondary Education for talks about strengths and weaknesses – and programs for improvement – in teacher education institutions and secondary schools.

7.5 Government implementation performance:The same group of key project preparation persons remained with the project as leaders throughout implementation. Only two project managers were used, one person with proven project management skills (to get the project going) and the other a person with deep understanding of the processes of both LPTKs and secondary schools (to provide substantive leadership). GOI provided sufficient counterpart funds in spite of the financial crisis at mid-project. One weakness, however, was the frequent late allocation of counterpart funds, caused by the fluctuating political and financial conditions during the crisis-ridden years of the project, meaning that some activities needed to be rushed (compressed into the last weeks of the fiscal year) and others (like dissemination workshops) curtailed.

7.6 Implementing Agency:Full support was given by the Central Project Implementation Unit consisting of education experts and professionals from education institutions, development teams, advisory committee, and sufficient number of dedicated administrative staff. The project was closely supervised by Director General of Higher Education and Director for Academic Affairs and, since 1999, the holder of a new GOI position, the Director of Teacher Education and Higher Education Personnel. The success of the project was due to adequate coordination by the project management and minimum intervention from the government as to project’s technical management.

7.7 Overall Borrower performance:In most areas the project delivered targeted outputs. An encouraging point is the efforts made toward sustainability. Many LPTKs have instituted plans to raise funds to continue school linkages, implement enriched student teaching, and continue school partnerships and action research. The main area of underperformance was the Borrower’s weakness in gathering evidence to demonstrate improved instruction and learning at the LPTKs and secondary school levels. The overall performance rating for the Borrow is satisfactory.

8. Lessons Learned

1. Innovation process. This project demonstrates how a relatively small investment (50 percent less than originally allocated) can generate useful breakthroughs and innovations in teacher education, a field generally considered to be conservative and resistant to change. The project's model of using centers of innovation to create and try out, on a limited scale, new curricula, student screening and enrichment programs, stronger linkages to secondary schools, improved student teaching approaches, action research related to instructional improvement, and inservice teaching turned out to be a cost effective way to stimulate new ways of doing things. Promoting widespread adoption of these innovations -- quite successful for some, not so for others -- was found to be a challenge in this era of growing institutional autonomy and clearly requires fresh thinking about the role of institutional accreditation and other inter-institutional development mechanisms.

2. Change model. Although the main development objective of the project was to enhance the teaching

- 16 -

learning processes in secondary schools through the improvement of teacher education, the project never clearly demonstrated where teacher education fits within a larger model of change (improvement) in teaching and learning. Future future projects would do well to describe such a model of change and describe how the other crucial determinants are also supported.

3. Outcome orientation. A major design weakness in the project -- one to be avoided in any further teacher training projects -- is its failure to effectively use outcome measures to document and assure project impact. Ultimately, efforts to improve teacher education should be judged in terms of improved knowledge and skills of training program graduates. Tracking these throughout a change program is means of quality assurance. The current project, with its "entry level assessment" laid down an important baseline for student change, but failed to use it effectively to demonstrate the impact of remediation and the overall impact of the teacher training through exit level testing. For practicing teachers, the inservice training that was offered, and the partnering with lecturers for action research and other professional growth opportunities, were never assessed in terms of observed changes in teacher performance; similarly those university lecturers who completed masters or doctors degrees were never called upon to demonstrate objectively improvements in their own instruction. In the future, teacher training programs need to have built-in mechanisms for tracking and assuring changes in valued outcomes.

4. Curricular revision of the kind envisioned in the Project -- geared to vertical, horizontal and external flexibility allowing graduates to teach at both secondary levels (lower and upper), to teach comfortably in more than one subject area, and to compete for jobs outside the field of education -- is a heavy and difficult task, more like structural reform than curricular revision. The project's efforts created some useful curricular materials but fell short of establishing such reforms, largely because they were focused too narrowly. The creation of curricular flexibility requires a system-wide effort to reduce curricular densities and rigidities -- a focus on learning scope and sequence not learning content. For instance, if the university were to stop teaching high school content to compensate for low student entry-level knowledge and instead were to institute mandatory remedial programs (tutoring) for underprepared students (see more below on ELA and 3S), it could create more space in the curriculum for flexibility. Such reforms cannot be done in isolation -- they need widespread support, even ownership, by major stakeholders to flourish. It would be useful to see them as part of the recently initiated efforts to create a competency-based (or perhaps better performance-based) curriculum.

5. The project's inservice teacher training was considered beneficial by both the tens of thousands of teachers who received it and the LPTK lecturers who provided it. Motivation was high among teachers in large part because it has helped them gain credits towards promotion. In fact, it is now clear that the most important feature of the project was not the funding of the training -- many participants seem willing to pay their own fees -- but the policy shift that allowed LPTKs to provide it. In exchange for this career boost for teachers the State has the right to expect improved teaching quality (see points in item 2 above). In the past the school system often regarded IKIP/FKIP training to be overly theoretical and largely irrelevant to the teacher's daily tasks. Given the Project's attempts to forge closer ties between the universities and the schools, has this changed? Have inservice graduates at the S1 level become more capable teachers? Some survey research at one institution showed that trainees had attained acceptable levels of knowledge and teaching skills, but there was no assessment of how much this was attributable to the course (no pre-post or control group comparisons). This is something that could be measured under DIKTI's transition project, perhaps using the observational instrument developed under the project (APKG). As implied in point 2 above, it would be important for any future inservice program to include among its performance indicators changes in teaching skills and classroom performance related to course participation.

Another lesson learned was that Project inservice training was much more accessible to those in major

- 17 -

urban areas (those hosting the LPTKs) than to rural teachers. New attempts must be found to reach those teachers largely (and unfairly) excluded from such training, especially if the country wants to keep teachers in rural areas. It may be time to allow LPTKs to develop programs of distance education, or at least to encourage different kinds of outreach, using remote study centers or holding special sessions during holiday breaks. Participation by rural teachers (especially where the teacher is the only income earner in the family) should be supported by scholarship funds. Scholarship support does not always have to come from DIKTI. There are growing numbers of examples of District Offices (Dinas) seeking and paying for teacher training. One promising idea connecting Dinas and the LPTK would be for certain field-oriented lectures to become resource persons to the teacher study circles or clusters (MGMP) set up by Dinas. 6. The higher than expected interest in action research, both that done by teacher-lecturer teams at schools and that done by lecturers in relation to their own teaching, reveals an emerging research culture in the schools and universities. Yet the agenda is still evolving. Research projects are still largely atomized and invisible. The few that have been published are published in academic journals of limited circulation. In the future DIKTI may want to develop a more user friendly format for reporting research findings, such as a digest or newsletter -- reporting not just individual findings but identifying trends and generalizable findings that can be used by clusters of teachers (MGMP) or teacher educators. Another unfinished part of the action research agenda is the action part. The two programs were established as a means of leveraging more effective teaching and learning processes. Since this has remained unmeasured (see point 2 above), it is not yet clear how successful this has been. During the transition project, DIKTI could set up means to assess using instruction improvement (in the schools the APKG could be used). In the university, instructional improvement by lecturers could be assessed through use of course evaluations done by students (more about that in the following paragraph). As in point 4 above, future projects should be sure to include such improvement among it performance indicators.

7. The project made a substantial contribution to improved qualifications of teacher educators through the masters and doctoral programs (domestic and overseas) that it funded. Its approach to enrolling lecturers in foreign universities (mostly European) using placement agents was both effective and economical (especially when the host country paid for the language training) and allowed the Project to exceed the target numbers (105 vs. 100) even under conditions of cost-cutting. However, the fields which were emphasized in the advanced studies were almost entirely in the disciplines, mostly science and math. As important as these are for the institutions, especially those expanding to the "wider mandate," the intended build up of senior scholars in education/teaching fields was overshadowed. (Whereas the SAR suggests an even balance between programs in the disciplines and those in pedagogy, the disciplines ended up with about 92 percent of the scholars.) Universities and DIKTI will need to take a careful look at future leadership potential in the teacher training field and put into place new incentives to build up the core of senior scholars in this field if it appears underrepresented.

Concerning those who have returned from advanced studies, both domestic and international, there was no attempt to measure whether they had improved their skills as teacher educators -- an issue parallel to that for teachers graduating from S1 inservicing (see point 2 above). Universities have the right to expect improved instruction from their upgraded lecturers. In many systems around the world this is accomplished through standard course evaluations done by students. The question is: do those who have received advanced graduate degrees teach any better than: a) they did before upgrading, and b) than those who have not? Such questions could be assessed during the new phase of PGSM and covered in any future teacher education projects.

8. Large scale efforts to modernize and equip science labs often show disappointing results. This Project purchased much equipment that was not used or was poorly maintained. If the objective of providing

- 18 -

well-equipped science labs is the more effective training of teachers, then it should not be necessary to obtain highly complex equipment, that difficult to operate and maintain. Instead there should be an emphasis on the same kind of equipment that will be used in the secondary schools. Also, project experience suggests that only those institutions which can demonstrate their capacity to use and maintain equipment should be provided such equipment in the future.

9. The Project's new approach to entry-level assessment (ELA), especially when connected to remediation through Student Support Services (3S), has proven to be a promising innovation but its potential has yet to be fully realized. A common complaint about school teachers is that they do not know the subject matter they are expected to teach. Project institutions developed the ELA in order to identify those prospective teachers whose subject matter mastery is weak. They, like all other students, are then offered tutoring to help them overcome deficiencies, but not all weak students take advantage of this. A more effective way to deal with the inadequate academic background of teacher trainees would be to make such tutoring mandatory -- a prerequisite to further study at the university. Such tutoring should be followed by retesting to make sure mastery is attained. This cycle of testing, remediation, and retesting is what quality assurance is all about. However, the ELA could also be used for quality control: given as a kind of licensing exam at the end to assure that teachers being sent to schools know what they are to teach. Those who fail at this point would not be licensed. The ELA used at the end like this -- as a post-test -- can also be used as a kind of program evaluation: demonstrating to management the program's success in improving mastery levels. This has been done to a limited extent in the Project and should be done more in the future to promote institutional accountability. The serious use of the ELA in the above ways suggests that the ELA itself should be of high quality. In the Project each institution was to develop its own version of the ELA, after receiving training. Clearly some institutions were/will be capable of building better tests than others. The serious use of the ELA suggests the need to make the best ones available to all institutions and the need to keep improving their quality. It may even be advisable for DIKTI to require some level of standardization.

10. The civil service status of teachers and university lecturers inhibits the effectiveness of quality improvement initiatives. The civil service incentive system does little to reward effective teaching, instead it encourages staff to join projects, do outside work (private school teaching and consulting), refrain from being controversial, and otherwise accumulate functional credits. The culture that thus pervades the university and the school is a bureaucratic not an academic one. Quality improvement in education requires higher levels of accountability than are now found in the civil service: accountability for performance in the classroom or lecture hall and for student outcomes, and real consequences if accountability standards are not met. This requires the assessment of teaching behavior (e.g., by students, by peers, or by supervisors), an awareness of student performance (e.g., proportion passing certain exams), and tenure and promotion decisions based on the results. Whether this kind of accountability can be accomplished by civil service reform or by taking teaching out of the civil service altogether is a question that needs to be confronted.

11. The conversion of all 10 public IKIPs to universities during the course of the Project has improved the status of teacher education in one sense, but potentially damaged it in others. University enrollment records indicate that cohorts of students admitted since institutional conversion are better qualified academically than before, in some cases putting them at par with neighboring well-established universities. Converted institutions report an improvement in the academic atmosphere on campus and higher levels of student expectations. However, it is not clear whether this conversion will improve the quality of teachers produced, since the higher achieving students will probably not chose teaching careers. Experience in other countries where this kind of conversion has taken place reveal the difficulty that the educational faculties have competing with higher status faculties for funds and resources. Under PGSM itself, many fewer of

- 19 -

the doctoral candidates than expected undertook studies in educational fields. During the "transition PGSM project" it will be important to study the extent to which the education track receives its fair share of resources, including staff time, space and equipment, library books, journals and other educational materials, fellowship opportunities, and annual budget allocations. It will also be important to track academic ability of those who select the teaching option, and to conduct tracer studies of graduates. Converted universities would be especially advised to develop strategic plans for maintaining the strength and status of teacher education. To assure continued quality improvement, it would also be desirable for all institutions to strengthen accountability at all levels (see point 6 on lecturers), including more rigorous accreditation procedures, involving peer and external reviews in addition to self-evaluation.

12. The new PGSM Project which can be considered a transition to full institutionalization of Project innovations can play a very positive role in further improving teacher education (see the many lessons learned above), but can also undermine sustainability. It can undermine sustainability if it continues to perpetuate a reliance on national funding for those activities which should be funded by the institutions out of their own budgets. This will require very careful consideration of what activities merit additional development funds and what should become routine. Care should be taken to avoid creating a sense of entitlement to Project funding -- institutions will only implement them if they get external funds. On the other hand, DIKTI should also avoid unfunded mandates, in which it requires the implementation of certain programs but fails to assess and arrange for financing.

9. Partner Comments

(a) Borrower/implementing agency:The Borrower's contribution to the ICR is in Annex 8.

(b) Cofinanciers:None.

(c) Other partners (NGOs/private sector):None.

10. Additional Information

- 20 -

Annex 1. Key Performance Indicators/Log Frame Matrix

Outcome / Impact Indicators:

Indicator/Matrix

Projected in last PSR1

Actual/Latest Estimate

- Student scores on project-developed tests at teacher training institutions.

Positive changes in observed student teaching skills in all (10) study programs at participating institutions (all but 2 statistically significant)

- Field experience program is running in all public teacher education institutions.

Yes, for all 30 institutions.

- Percentage of faculty of teacher education institutions with a post-bachelor's degree.

2001 figures: 51.5%; if faculty in pipeline counted: 63.2%.

- Number of graduates from public teacher education institutions employed.

Government freeze on hiring new teachers in place during most of the project

Output Indicators:

Indicator/Matrix

Projected in last PSR1

Actual/Latest Estimate

Component 1. Improved LPTK pre- service and in-service programs.

1.1. English, social sciences and subject specific curriculum revised; pedagogy courses developed; double major and major-minor programs offered.

a. All of the three curricula have been designed and pre-tested in 8 LPTKs (English), 3 LPTKs (Social Sciences), and 3 LPTKs (Subject Specific Pedagogy).b. Flexibility programs, including double majors and major-minor, have been designed in English and Social Sciences curricula but they have not been implemented yet since these programs are meant to be offered in last semesters (after the students have completed all or most of their first major programs).

1.2. Science and math laboratories renovated and equipped; storage facilities installed and used; science workshops for LPTK lecturers in operation.

a. Biology, Physics, Chemistry and Mathematics Laboratory equipment in accordance with the Basic Science Team Standard have been purchased and installed in 12 LPTKs (Jakarta, Bandung, Semarang, Yogyakarta, Surabaya, Malang, Singaraja, Makassar, Manado, Gorontalo, Padang and Medan). Science Laboratory in UNJ had been renovated and used for S1 students’ practicum.b. Language lab equipment have been purchased and installed in UM, UNESA, UNNES, UNEJ, UNRAM, and IKIP Singaraja. Language lab. equipment as a supplement to the existing lab. have been provided to UNRI and UNP. All language lab. Equipment has been used to support the implementation of the Newly English Curriculum developed by the SSTD English Development Team.c. Science Laboratory Storage for the equipment has been installed and used (UNESA, UNIMED).

1.3. Learning materials selected, revised and piloted, and further revised.

a. Twelve (12) subject specific pedagogy areas have been identified, basic course outline have been reviewed and developed for

- 21 -

each subject area, b. 120 modules have been written, reviewed, and revised and used for distance learning program by the Open University.

1.4. Number of staff offices, conference rooms, student-faculty and inter-faculty interactions, and number of faculty attending meetings and seminars increased.

a. Renovation/Remodeling of staff offices and conference rooms have been built in 12 LPTKsb. Student-faculty consultation meetings increased in some LPTKs. Faculty-faculty interaction also increased accordingly. c. Number of seminars were conducted and attended by faculty members and experts from local and international Universities.

1.5. Secondary text books purchased and education references materials developed, and distributed to LPTKs.

a. Lower and Upper Secondary School Textbooks purchased and distributed to all public LPTKs. The books have been used by students who are preparing for teaching practice in secondary schools and lecturers. The books are also used by secondary school teachers to develop their own teachings.b. Biology, Chemistry and Physics Textbooks have been translated from English to Indonesian published and distributed to all LPTKs except UNPATTI due to civil disturbances in the province of Maluku. c. Eight of twenty-two textbooks produced by refresher program participants have been published, purchased and distributed to all public LPTKs.d. English Student Workbooks, Teacher Guide-Book and Audio Cassette were developed and distributed to 8 LPTKs.

1.6. Student Support Service centers established and operating, based on students entry level assessment scores.

a. Student Support Centers were established at 30 LPTKs.b. National Seminar was held in Yogyakarta in April 2001 for 3S program, reporting results of study in 14 LPTKs of effectiveness of 3S.c. Students having low Entry Level Scores (Mahasiswa ELA Rendah =MER) have been identified in 7 LPTKs.d. Instruments for detecting learning problems have been developed and administered in 30 LPTKs.

1.7. Number of scholarships awarded. a. 18,725 scholarships for upgrading qualifications of secondary teachers from D3 to S1 were awarded, exceeding project target. Upgrading of 600 teachers through the Open University was cancelled due to cost containment. b. Studies on second and last semester of final cohort were completed in September 2000. Performance evaluation of 15 LPTKs was presented in national seminar in March and December 2000c. A study comparing teachers with regular S1s and those receiving S1 upgrading through the project was conducted and results results presented in a National Seminar.

- 22 -

Component 2. LPTK partnerships with secondary schools established.

2.1. Activities between LPTK and schools, numbers of teachers joining on-campus programs, numbers of lecturers working in schools, numbers of LPTK graduates being hired in schools, and numbers of joint research between LPTK and schools.

a. 912 lecturers from 11 LPTK have been actively working in secondary schools contributing to teaching, action research, student counseling, learning material improvement, and community services.b. Field Experience Program (PPL): -180 partner schools were appointed, handbook, and manual for PPL were developed. -Cooperating teachers and supervising lecturers were trained. -Tryout of new PPL models were conducted by LPTKs and partner schools in all provinces.c. Classroom-Action Research: -5 Batches of CAR were implemented. -559 proposals were selected and implemented. -National Seminars were held and findings were reported. -National evaluation of CAR was conducted and results were presented in national seminar in December 2001.d. Two FKIPs initiated pilot deployment using own funds (Tadulako in Sulawesi, Mulawarman in Kalimantan): -25 action-based research were conducted by participating deployment staff.

2.2. International visits conducted, and lessons applied to strengthening LPTK-school linkages.

a. International visits were conducted during the first two years (before this program was cut) followed by in-country workshops to disseminate and plan further activities in building LPTK-School linkages in Indonesia.b. Some LPTKs (UPI, UM, UNESA, IKIP Singaraja, UNM) continued conducting significant activities with their respective international partners (e.g., OSU & La Trobe) after program funding was cut.

Component 3. Improved quality of LPTK teaching staff.

3.1. Number of domestic and international fellowships for Masters program granted.

1301 and 57 teacher educators were awarded domestic S2 program fellowships and international Masters program fellowships respectively.

3.2. Number of domestic and international fellowships for doctorate program granted.

16 teacher educators with S2 qualification were awarded Domestic Doctoral program fellowships and 105 awarded international Doctoral program fellowships.

3.3. Number of international refresher courses accomplished.

22 teacher educators from selected LPTKs attended the non-degree professional development through refresher program.

3.4. Number of course-related materials produced by returnees of refresher courses.

22 course-related materials were produced by the returnees of refresher program, but only 8 materials were revised, published and distributed to all public LPTKs.

3.5. Number of regional workshops conducted and subject specific pedagogy courses taught in the workshops .

No information.

- 23 -

Component 4. Building Education Research Capacity.

4.1 Assessment instruments developed and administered, results analyzed and presented to the 3S, and teaching adjusted accordingly.

a. Diagnostic entry level assessment instruments (ELA) were developed in 10 LPTKs.b. ELA results were analyzed and scores has been presented to the Student Support Services (3S) Centers.c. Exit-level content knowledge of LPTK students was assessed using the ELA. d. Quality Assurance using ELA was conducted in 4 LPTKs (UM, UNM, UNESA and UNP); using the ELA and measures of teaching performance was conducted in 2 LPTKs (UNESA and UNP).e. The results of such assessments were reported to the Directorate General of Higher Education, MONE and to program coordinators and lecturers from 18 and 2 STKIPs.f. Training in use of ELA and Quality Assurance (ELAQA) 4 academic fields was conducted for 30 LPTKs.

4.2. Number of student teachers assessed after PPL and each LPTK student assessed at graduation.

Quality Assurance using and measures of teaching performance was conducted in 2 LPTKs (UNESA and UNP).

4.3. Number of grants awarded for proposals, RII designed and implemented, results used and published.

229 research proposals on RII were accepted for funding, National seminars on findings were held in 1999, 2000 and 2001.

Component 5. Preparing for a Wider Mandate.

5.1. Guidelines for wider mandate drafted, letter of intent and proposals submitted, visitation and LPTK selection made, and program planning processed.

a. Guidelines, letter of intents, proposals, visitations, and program planning were set up and completed, not only for wider mandate but for converting LPTKs to universities.b. At the moment, ten public IKIPs have now become universities and two public STKIPs have now become IKIPs.

5.2. Meetings with BAN conducted, standards for LPTKs adjusted, self-evaluation and peer validation implemented, evaluation report writers and peer validators trained, and seminars for system improvement conducted.

5.3. Career information office available, data base of career information developed and shared, and number of students selecting double major and major-minor.

a. Meetings with BAN were organized, guide book for self evaluation was completed, and its dissemination to all LPTKs was conducted.b. The DGHE stressed the importance of conducting self evaluation and demanded the self evaluation reports be attached in LPTK annual budget proposals.

a. Career information services were set up in UPBK (Guidance and Counseling Imple. menting Unit).b. List of information have been developed regarding types of competencies, occupations, vacancies, and training in non-education areas relevant to LPTK students.c. Data regarding number of students selecting double majors/major-minor programs have not been available yet

1 End of project

- 24 -

Annex 2. Project Costs and Financing

Project Cost by Component (in US$ million equivalent)AppraisalEstimate

Actual/Latest Estimate

Percentage of Appraisal

Component US$ million US$ million1. Improving Preservice & Inservice Teacher Education 30.602. Strengthening Linkages to Secondary Schools 5.403. Raising the Qualifications of Teacher Educators 32.404. Building Educational Research Capacity 2.905. Preparing for a Wider Mandate 1.106. Project Implementation 4.20

Total Baseline Cost 76.60 0.00 Physical Contingencies 2.40 Price Contingencies 8.10

Total Project Costs 87.10 0.00Total Financing Required 87.10 0.00

The Government was asked to supply information on actual project costs by component but did not respond. Bank systems only track expenditure by disbursement category. Hence, portions of this annex could not be completed.

Project Costs by Procurement Arrangements (Appraisal Estimate) (US$ million equivalent)

Expenditure Category ICBProcurement

NCB Method

1

Other2 N.B.F. Total Cost

1. Works 0.00 2.80 1.20 0.00 4.00(0.00) (1.40) (0.60) (0.00) (2.00)

2. Goods 2.80 7.60 4.60 0.00 15.00(2.10) (5.30) (3.20) (0.00) (10.60)

3. Services 0.00 0.00 1.60 0.00 1.60(0.00) (0.00) (1.10) (0.00) (1.10)

4. Learning/Teaching 0.00 1.20 0.80 0.00 2.00Materials (0.00) (0.90) (0.50) (0.00) (1.40)5. Fellowships 34.7 34.7

(27.3) (27.3)6. Grants 5.9 5.9

(4.1) (4.1)7. Scholarships 10.0 10.0

(7.0) (7.0)8. Local Training 0.00

(0.00)0.00

(0.00)9.4

(4.7)0.00

(0.00)9.4

(4.7)9. Project Operating Costs 0.00

(0.00)0.00

(0.00)4.5

(2.2)0.00

(0.00)4.5

(2.2) Total 2.80 11.60 72.7 0.00 87.1

(2.10) (7.60) (50.7) (0.00) (60.4)

- 25 -

Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent)

Expenditure Category ICBProcurement

NCB Method

1

Other2 N.B.F. Total Cost

1. Works 0.00 0.00 0.00 0.00 0.00(0.00) (1.06) (0.00) (0.00) (1.06)

2. Goods 0.00 0.00 0.00 0.00 0.00(0.00) (7.56) (0.00) (0.00) (7.56)

3. Services 0.00 0.00 0.00 0.00 0.00(0.00) (0.00) (0.49) (0.00) (0.49)

5. Fellowships 0.00 0.00(11.77) (11.77

6. Grants 0.00 0.00(2.69) (2.69)

7. Scholarships 0.00 0.00(2.41) (2.41)

8. Local Training 0.00(0.00)

0.00(0.00)

0.00(1.11)

0.00(0.00)

0.00(1.11)

9. Project Operating Costs 0.00(0.00)

0.00(0.00)

0.00(1.35)

0.00(0.00)

0.00(1.35)

Total 0.00 0.00 0.00 0.00 0.00(0.00) (8.62) (19.82) (0.00) (28.44)

The project implementation office was not able to provide the overall project costs by expenditure categories therefore, the above table only reflects loan expenditures. The loan expenditures were calculated based on data available as of June 23, 2003 in the Integrated Controller's System. After project restructuring and funds cancellation in 1998 the total loan amount was reduced to $30.4 million. As of June 23, 2003 an additional US$1,959,571.81 will be cancelled leaving a total loan amount of US$28,440,372.

Project Financing by Expenditure Category (in US$ million equivalent)

Expenditure Category Appraisal Estimate Actual/Latest EstimatePercentage of Appraisal

Bank Govt. CoF. Bank Govt. CoF. Bank Govt. CoF.Civil Works 2.00 2.00 1.05 52.5 0.0Equip/Furn/Materials 12.00 5.50 7.56 63.0 0.0Consultant Services 1.10 0.50 0.49 44.5 0.0Fellowships Domestic 7.40 7.40 2.92 39.5 0.0 International 19.90 0.00 8.82 44.3 0.0Scholarships 7.00 3.00 2.41 34.4 0.0Grants 4.10 1.80 2.69 65.6 0.0Local Training 4.70 4.70 1.11 23.6 0.0Project Operating Costs 2.20 2.30 1.35 61.4 0.0

The Staff Appraisal Report only provides project financing data by expenditure category. The data used in calculating the Bank's portion of actual expenditures comes from the Integrated Controller's System - June 23, 2003. The project implementation office was not able to provide the actual figures for the GOI contribution.

- 26 -

Annex 3. Economic Costs and Benefits

Not applicable

- 27 -

Annex 4. Bank Inputs

(a) Missions:Stage of Project Cycle Performance Rating No. of Persons and Specialty

(e.g. 2 Economists, 1 FMS, etc.)Month/Year Count Specialty

ImplementationProgress

DevelopmentObjective

Identification/Preparation08/1994 4 Education Specialist (Mission

Leader); Teacher Education Specialist (Consultant); Science Education Specialist (Consultant); General Educator (Consultant).

11/1994 4 Education Specialist (Mission Leader); Teacher Education Specialist (Consultant); Operations Officer; Technical Support.

02/1995 1 Economist (Task Manager)09/1995 4 Economist (Mission Leader);

Senior Education Specialist; Education Specialist.

Appraisal/Negotiation06/1995 7 Economist (Mission Leader);

Education Specialist; Research Assistant; 3 Educators Consultants; Operations Assistant.

09/1995 4 Economist (Mission Leader); Senior Education Specialist; Operations Analyst; Education Specialist (Consultant).

07/1996 3 Education Specialist (Mission Leader); Teacher Education Specialist (Consultant); Education Specialist (World Bank Resident Mission).

Supervision03/1997 3 Senior Education Specialist

(Mission Leader); Teacher Education Specialist (Consultant); Education Specialist.

S S

09/1997 3 Senior Education Specialist (Mission Leader); Teacher Education Specialist (Consultant); Education Specialist.

S S

09/1998 3 Senior Education Specialist (Mission Leader); Teacher Education Specialist

S S

- 28 -

(Consultant); Economist.03/1999 3 Economist (Mission Leader);

Teacher Education Specialist (Consultant); Education Specialist.

S S

08/1999 3 Education Specialist (Mission Leader); Teacher Education Specialist (Consultant); Education Specialist.

05/2000 2 Economist (Mission Leader); Senior Education Specialist.

05/2001 2 Education Specialist (Mission Leader); Teacher Education Specialist (Consultant).

S S

ICR03/2002 5 Senior Education Specialist

(Mission Leader); Teacher Education Specialist (Consultant); Education Specialist; Procurement Consultant; Financial Management Advisor.

S S

(b) Staff:

Stage of Project Cycle Actual/Latest EstimateNo. Staff weeks US$ ('000)

Identification/Preparation 343,353Appraisal/NegotiationSupervision 336,568ICRTotal

- 29 -

Annex 5. Ratings for Achievement of Objectives/Outputs of Components(H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable)

RatingMacro policies H SU M N NASector Policies H SU M N NAPhysical H SU M N NAFinancial H SU M N NAInstitutional Development H SU M N NAEnvironmental H SU M N NA

SocialPoverty Reduction H SU M N NAGender H SU M N NAOther (Please specify) H SU M N NA

Private sector development H SU M N NAPublic sector management H SU M N NAOther (Please specify) H SU M N NA

Increasing social status of teacher education

- 30 -

Annex 6. Ratings of Bank and Borrower Performance

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory)

6.1 Bank performance Rating

Lending HS S U HUSupervision HS S U HUOverall HS S U HU

6.2 Borrower performance Rating

Preparation HS S U HUGovernment implementation performance HS S U HUImplementation agency performance HS S U HUOverall HS S U HU

- 31 -

Annex 7. List of Supporting Documents

Project Information Report (PIR)Staff Appraisal Report (SAR)Loan AgreementMemorandum of the PresidentMinutes of NegotiationsMission Aide-memoiresBack-to-Office ReportsProject Status Reports (PSRs)Project Correspondences

- 32 -

Additional Annex 8. Borrower's Contribution

Programs That Went Well

Almost all programs of the SSTD have achieved satisfactorily most of the key targets and have affected institutional changes within the context of improving internal efficiency, productivity of staff, school teachers, LPTK’s students, and providing linkages between LPTK and secondary schools. Each program and activity, that went well, will be discussed separately in terms of its original component as follows.

Component 1: Quality Improvement subsumed several programs, namely: curricula development, educational facilities (office spaces, learning materials, references, science and language labs, student support services and career planning development and inservice training.

A new English Curriculum with its intensive course materials and strategy was established satisfactorily. The teaching materials for students and teachers were produced, distributed, and tried out in 8 LPTKs. This program went very well due to the fact that the program was relatively well managed by highly qualified and fully committed personnel, especially of the Intensive Course Team members. The Social Studies and the Subject Matters Specific Pedagogy Curricula are also completed in due time. Each of the curriculum retains its own characteristics.

Concerning horizontal flexibility, both English and Social Sciences curricula had designed portions of courses and their credit allocations for students from other departments who want to take English and Social Sciences as their minor programs. These plans have not reached their implementation yet since:

(a) the new curricula were meant for students entering the LPTKs at the time the new curricula were introduced; (b) the departments (jurusan) tend to allow their students to take minor programs during their last semesters, when students have or almost have completed their own major programs. It is realized, however, that the work of curricular flexibility is concerned with the entire system of educational institution rather than with particular curriculum areas, such as English, Social Sciences, or Subject Specific Pedagogy. In conjunction with the expectation specified in SAR, both English and Social Sciences curricula could only design courses for minor programs as providers.

The vertical flexibility has been developed and implemented through PGSM curriculum development, by means of:

(a) Providing courses in English and Social Sciences which would also prepare LPTK students to teach in junior secondary schools. The introduction of common courses in IPS, for example, was meant to provide both horizontal and vertical flexibility since in junior secondary schools students do not learn history, geography or economy but social studies.(b) Expanding teaching methodology contents covered in Subject Specific Pedagogy courses to accommodate the instructional needs for both senior and junior secondary schools.(c) Requiring all LPTK students to teach in both senior and junior secondary schools during their field experience programs (PPL).

Aspects of 3S, which were believed to have been successfully implemented, are the development of AUM (student’s learning problems probing instruments), training of tutors, implementation of peer tutoring, and the establishment of linkages to Guidance and Counseling Implementation Units. Prior to the actual implementation, the implementers were trained in using the instruments and giving advisement of students who faced learning problems. Career Planning Development was also successful in terms of its success in

- 33 -

informing educational and non-educational job opportunities, and training for career information tutors .

In-service teacher education was conducted well, even it exceeding the project target. Many of school teachers, especially in those remote areas have not had the chance to join the in-campus in-service education. This is due to the design of the project which did not provide transportation and living allowances to teachers. However, after the project completion they began entering the program on their own expenses. Studies on second and last semesters of the final cohort were completed in due time. A comparative study between regular S1 graduates and inservice S1 graduates was also conducted. Learning materials in the form of modules (120) were developed, reviewed, revised, and re-written, for use by Open University students.

Purchased science equipment and language laboratories have been used by S1 students (both regular and in-service teacher programs) and by LPTK’s staff who joined the Academic Staff Deployment program. At least 4 LPTKs have developed lab practicum manuals through the financial assistance from JICA. Renovated and remodeled staff offices, conference rooms and storage rooms for equipment have been used for preparing courses and student consultation.

Component 2: School Linkages subsumed several programs, namely: field experience program, academic staff deployment, and classroom action-based research. Parts of the field experience programs that particularly went well were: criteria for teacher and faculty supervisors, guide books to field experience, the student teachers performance instruments, and training for teacher and staff supervisors. Such successes were due to the implementation of the synergised bottom-up and top-down work strategy.

Action-based researches were successfully implemented in a collaborative way and results were used to improve teaching-learning processes in secondary schools. The action-based research was well planned and managed. The program was designed initially with training, development of terms of reference, nomination of research proposals, selection, implementation, and ending with evaluation.

For most of the LPTKs, international partnership have been quite successful, some continued their partneship after program termination.

The ideas of Academic Staff Deployment was initially resisted, but was finally well accepted and implemented by nine LPTKs. Various activities were held in schools through a collaboration between school teachers and LPTK’s educators. The benefit gained by the deployed staff is deepened understanding about the schools. The benefits gained by the teachers are: (1) improved teaching methods, and (2) improved learning materials. The students also benefitted in that they were able to do lab experiments using equipment from LPTK.

Component 3: Staff Development subsumed several programs, namely: masters and doctoral programs and refresher program. Masters and doctoral programs were both successful not only in terms of optimum achievement (number of graduates) but also in terms of qualitative achievement, such as improved accreditation for the institution, improved ability in conducting research, and improved recency of learning materials. Based on data gathered through questionnaires, graduates returned to their original institution and were assigned to teach S1 or S2 students in accordance with their academic background. The international refresher program was undertaken successfully, but due to budget cut only one batch was implemented. Component 4: Education Research subsumed three programs, namely research for the improvement of instruction, entry-level assessment, and quality assurance. Parts of ELA that went well are the construction,

- 34 -

administration, analysis of tests or instruments, the use of ELA results by tutors and dissemination of ELA items to LPTKs. Parts of quality assurance (QA) that are mostly successful are the construction, administration, and dissemination of test items. Quite a number of research were conducted by skilled lecturers and the results of which were used for the betterment of teaching-learning processes at LPTKs. Systematic mechanism was applied to assure the selection of good research. Moreover, it was followed with intensive monitoring and evaluation of reports. The best research results were presented in the national seminar and published in education research journals.

Component 5: Wider Mandate subsumed two major programs, namely: wider mandate and accreditation. Ten LPTKs were finally converted to universities, while two STKIPs were converted into IKIPs. The converted IKIPs were given permission to widen their primary mandates to run the non-education programs without neglecting their primary mandate (teacher education) The conversion yielded positive effect on the quality of applicants. The accreditation program was conducted through self evaluation only. DGHE requires LPTKs to conduct self evaluation and assists them financially by including funding for such in the LPTK’s annual budget.

Programs That Didn’t Go Well

Component 1: Quality Improvement . Parts of curricula development that is felt lacking are its dissemination and legalization from DGHE in order to secure implementation in all LPTKs due to the new ministerial policy on university autonomy in curriculum development. An aspect of science equipment provision still considered problematic is maintenance. This is due to budget cut and non-availability of qualified personnel. Every room of the renovated staff offices has been shared by 2 to 4 staff members, and this has negatively affected the effectiveness of the consultation activities (though it may have a positive affect in terms of efficiency).

Component 2: School Linkages. In the first two years of Project’s implementation, International Partnerships program was implemented successfully, however, there were certain LPTKs and their international partners which were not able to carry out the programs successfully due to lack of communication and mismatches in the programs. This program was terminated due to drastic budget cut.

Component 3: Staff Development. Refresher Program was only successful in sending 22 teacher educators out of 150 targeted by the Project to do the non-degree professional development in international universities. Only 8 out of 22 manuscripts produced by the returnees have been printed and distributed to all LPTKs. Some of the drafted manuscripts produced by the returnees were not approved by the expert reviewers. There was no agreement by both parties on which parts needed revision.

Component 4: Education Research. The Entry Level Assessment results were not used widely in quality assurance due to time limit. ELA was conducted and the results were used for tutoring students in the third year of the Project’s implementation. Therefore QA can only be administered in semester VI rather than at the end of semester VIII. Data on entry and exit levels has not been obtained as a form of quality assurance.

Component 5: Wider Mandate. This program that went far beyond its expectation is the conversion of IKIPs into universities and STKIPs into IKIPs. Although there is an increase in the quality of applicants but the conversion also recognized that the institutions’ teacher education mission is under threat in certain ways.

- 35 -

Sustainability

Almost all programs of the SSTD have achieved satisfactorily most of the key targets and have affected institutional changes, particularly the internal efficiency, productivity of staff, school teachers, and LPTK’s students. It also created amicable linkages between LPTK and secondary schools. Therefore, most of the Project’s actual outputs and outcomes are envisioned as most likely sustainable. There are two main reasons why the ex STTD’s programs would be sustained in the future. Firstly, the Project has designed a gradual decrease in terms of funding of all programs and activities. This scheme pushes the LPTKs to allocate budgets for sustaining previously conducted programs and activities. Secondly, the establishment of a new directorate within DGHE, that is, the Directorate for Supervision of Educational Staff Development and Personnel of Higher Education (Direktorat Pembinaan Pendidikan Tenaga Kependidikan dan Ketenagaan Pendidikan Tinggi) assures the sustainability of the truly beneficial programs. This new directorate is committed to sustaining some of the SSTD’s programs and activities.

In light of DGHE policy, efforts have been made for ensuring sustainability of most programs through the combining of bottom-up and top-down implementation of those programs and activities. In each institution, two junior and highly active staffs were appointed by DGHE to help plan future programs and activities. Since the beginning of the SSTD, some programs are integrated in each institution’s Strategic Planning (Renstra). By such a policy, the programs and activities are to be sustained in the future. Some programs were jointly planned between LPTK and the Regional and Provincial Office of National Education (Dinas Diknas), such as action-based research and field experience program (PPL).

Fundamentally, almost all programs included in the five components are already being sustained. The sustainability of all programs was made possible through their insertion into the existing and relevant offices or boards (LEMLIT, UPBK, BKSK, etc) within its own institution. The new management strategy that will be implemented in the future is to combine the top-down and bottom-up approach in order to secure sense of belonging and responsibility by the respective LPTK and schools. The funding which was originally derived from the SSTD has been replaced by the institution’s routine, recurrent, and annual budgets (for examples: DIKs, DIP, BP3, DUE-Like, BPPS, etc.). Funding has also been obtained from local governments.

There are some programs that will definitely be continued and sustained. The programs expected to be sustained are: the action-based research, academic staff field assignment, in-service education for secondary school teachers, entry level and exit level assessment, student support services, quality assurance, self evaluation program, field experience for student teachers, curriculum dissemination, research for the improvement of instruction, books and references, and learning materials production.

Lessons Learned

Positive Lessons Learned. Evaluation reports of both partnerships and classroom action research (2001) indicated the importance of combining bottom-up & top-down approaches in enhancing program dissemination. Reports of RII (2000) and classroom action research (2001) noted that competitive strategies had increased quality as well as institutional participation in respective programs. Report of the English language curriculum tryout (2000) noted the effect of the intensive course approach (IC) in improving student learning within the first semester in LPTKs. Annual Project Reports also suggested that early attention on program sustainability makes LPTKs more aware of this issue resulting in concrete efforts to integrate the programs into the existing organizational units. Increased number of teaching facilities have improved the teaching-learning processes at LPTKs, which in turn will affect the quality of graduates. Sufficient offices for staff have improved interactions between student- staff, and staff-staff. In

- 36 -

order to publish learning materials, it is certainly necessary to establish a link between LPTKs and local and national publishers.

Negative Lessons Learned. Reports of the 3-S (2000) noted the lack of coordination among ELAQA, 3S and the implementing study programs in certain LPTKs, minimizing potential effect of the program. Report of the CPD program also suggested the need of building better coordination between CPD and faculties and study programs in each LPTK. Report of the science equipment monitoring in several LPTKs (2000) noted the lack of training in the use of new equipment for teaching purposes, minimizing the benefit of that investment. Other reports -- PPL survey, 2001 and annual PGSM coordination meeting – point to the following lessons: (1) the problem of imposing one national model of PPL for all LPTKs with varied conditions, and (2) the potential risk of rapidly converting IKIPs to universities from the viewpoint of teacher education development. There was not enough thought given to the staff development planning in accordance with the needs in the LPTKs.

Some of the above lessons have been considered relevant and applicable for similar projects i.e. (1) combining bottom-up and top-down approaches in introducing new ideas; (2) the role of competitive strategies in increasing both quality and program participation including research capacity; (3) the need for giving early attention on the sustainability of the programs; (4) the problem of imposing one national model for all institutions with varied background and conditions; and (5) the importance of training in use of new equipment for teaching purposes.

Bank’s Performance

The Bank had provided regular and effective supervision through the establishment a consistent mechanism and process in loan agreement. The Bank had also satisfactorily provided the CPIU with adequate assistance prior, during and after the project’s implementation. The rating standards and procedures were well established and effective throughout the span of the Project. However, there was minor administrative shortcoming as the Bank did not provide feedback of the matters requested or shared with the Bank. The Bank’s overall performance is rated satisfactory.

- 37 -

- 38 -