Document ofThe World Bank

FOR OFFICIAL USE ONLY

Report No: 23742

IMPLEMENTATION COMPLETION REPORT

IThXT A

lNJO t*i

1 C. A fTL'.JtUN

Al rrr . a - - --r -np T ,ln I W -l TjVLTIIN T1lr. P"V1'JUi'I 'JLr Ualb30.j 1VilL~1'JP

IlN'DOINESIA

FOR ANINDUSTRIAL TECHNOLOGY DEVELOPMENT PROJECT

June 26, 2002

Private Sector Development UnitEast Asia and Pacific Region

This document has a restricted distribution and may be used by recipients only in the perforrnance of theirofficial duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CTUTRRENCY EQTTTVATLENTS

(Exchange Rate Effective December 31, 2001)

Currency Unit = Indonesian Rupiah (Rp)D-I.1 AA = UTC$ 0.00AA0

US$ 1.00 = RplO412.50

FISCAL YEARA.rIJL 1I -LVidichI J.l JLt 1-y J- JUlLK J3

A 1-)n Tnl TT A ',Tf'.TC' A kITr% A f'fl TXx 'kOPLDDJXJ V It% I%JIN) at1L'LJ 1I1%X .JI 1 lvi

B!Sm ~ Sernarar. ncyLldustria!I RkT) TnsetitelBPPIP Agency for Industrial R&DDDDT A ge.cy fnr 'h. Asccessm.nt n.d Ar,nninatinn ofTP'hno%nImu

CAS Country Assistance StrategyCE'TP A ?T, C P,h,.,.,nt for TPhn,n1nm, Pnli,py Analirsie anut macmncam.mpnt

CSIRO Commonwealth Scientific and Industrial Research OrganizationAN PA I reAA gI,*lf Sen.ices ,..r i. 1 A ,$n-.tSce

(Dana Kemitraan Peningkatan Teknologi Industri)

IRDCI Institute for Research and Development for Chemical IndustryII LJT IUULIMLI ial I nds ULIrLaY Tvelo,;IVJ.nen,i I IVJ.cL

KIM National Metrology CenterK' A CTY-A-- - -- A Cor ..,c Tad-Adu y-.

KIST Korea Institute of Science and TechnologyLr HIGrlublneil UIrS'u:'ILIe VI Sciences

MBIS Management & Business Integration System

MOi Ministry of Industry (later becamne MOIT: Ministry of Industry and Trade)MvSS Mvanage-me-nt SysLteriU .2eL-guieuiiig

MSTQ Metrology, Standards, Testing and QualityPAU Policy Advisory UnitPMRC Project Management Resource CenterPPMS Project Process Management SystemsPTSIs Public Technology Support InstitutionsPUSTAN Center for industrial StandardsQAG Quality Assurance GroupRDCAC Research and Development Center for Applied ChemistrySMIs Small and Medium-sized IndustriesTA Technical AssistanceTSO Technical Service Office

Vice President: Jemal-ud-din KassumCountry Manager/Director: Mark Baird

Sector Manager/Director: Amanda S. Carlier, ActingTask Team Leader/Task Manager: Lily Uy Hale

INDONESIAI1NiDUSTRlAL TECIrIiNOLOG'Y DE-v'ELOffi;NT PROJIECT

CONTENTS

Page No.1. Project Data 12. Principal Performance Ratings 13. Assessment of Development Objective and Design, and of Quality at Entry 24. Achievement of Objective and Outputs 65. Major Factors Affecting Implementation and Outcome 136-n.-Sutainabilitv!S7. Bank and Borrower Performance 168. LessonsI £0 .. edC.LLt 10

9. Partner Comments 1910. Additionai information 21Annex 1. Key Performance Indicators/Log Frame Matrix 22Annex 2. Project Costs and Financing 25Annex 3. Economic Costs and Benefits 27Annex 4. Bank Inputs 28Annex 5. 'Ratings for Achievement of Objectives/Outputs of Components 30Annex 6. Ratings of Bank and Borrower Performance 31Annex 7. List of Supporting Documents 32

Evaluation Summary

The Industrial Technology Development Project (ITDP) was triggered by the findings of many studies inthe early 1990s about the importance of technology support services in enhancing the competitiveness offirms, especially SMIs on the one hand, and the weaknesses of public technology support institutions inIndonesia (which employed about 80 percent of the country's scarce scientific, technical and engineeringmanpower), on the other hand. The series of events led to the formulation of a project which becameeffective on March 18, 1996, with an estimated project cost of US$60.9 million and a loan amount ofUS$47.0 million. However, in light of following the 1997 economic crisis, the Government of Indonesia(GOI) launched a series of debt portfolio restructuring across the board which impacted all IBRD loans,including the ITDP on three separate counts. These events led to the scaling down of the ITDP loanamount by US$8.5 million to US$38.5 million. At project closing on December 31, 2001, the loandisbursement amounted to US$32.64 million, i.e. 84.7% of the reduced loan amount.

Overall Ratings

The project was rated satisfactory from project design, through preparation, supervision and its impact tothe SMIs. Throughout project implementation, Bank staff worked closely with the government and theimplementing agencies and provided them with extensive assistance. During the project life, the changes oftask managers occurred three times which may have caused delay in processing of documents in thetransition period. However, taken in the context of overall performance of the team, the Bank performancewas rated satisfactory.

The commitment of the Government (despite several political changes throughout project implementationperiod) to the project was shown through the continued allocation of operational counterpart funding to theproiect entities. This has helped to preserve the original objectives of the proiect components,notwithstanding the economic crisis experienced by the country in the late 90s. Although there wereinstances where the borrower performance could have been better. e.g., the unnecessarily prolongednon-response to some of the correspondence delayed document processing, in the context of overallperformance and in terms of achieving the project objectives, the Borrower's implementation performancewas rated satisfactory.

Project Objectives

The objective of the nroiect (ITDP) to enhance the comnetitiveness of Indonesian industry. narticularlv thesmall and medium-sized industries (SMIs) was consistent with the Bank's 1995 Country AssistanceStrategv (CAS). which emphasized the importance of imnroving Indonesia's competitiveness and the needfor a shift towards a productivity-based economy and the complementary private provision of supportinginfraqtnictnre services. The nroiect sunnorted this CAS objective by focusing on the improvement oftechnology support institutions (R&D centers) and facilitating the access of SMIs to these technologyservices which were designed to increase their competitiveness througrh improved manufacturing and

management skills.

The project was responsive to the needs of the Borrower. It envisioned different benefits such as yieldingpeconr%mic arnd firannaren1 r nhim tn 1ndnnP-qia'Q SMT mnmifarh.hiiring sector hnAnri ~inghtnntin2 catalytic eff-ect

on other public sector investments in the provision of technology services, as well as influencing Indonesia'snational teChnolong nnlicieanc nn nrorams. Thscp hPnefits arP Penxrpter to hb reli7Pe thhronuh the

implementation of the project components which included the following: (a) providing public and privatetechnology support services to the SMIs; (b) facilitating their access to public and private technologyservice providers (through a DAPATI Program which provided a matching grant scheme to encourageSMIs to avail of the technology services aimed at increasing their competitiveness); (c) strengthening andmaling more market-oriented the public technology support institutions; and (d) improving the formulationand coordination of industrial technology policies.

Summary of Findings

The project enabled the GOI to take advantage of the lessons from Bank-supported technology projectsthroughout the world, take a more proactive stance in focusing industrial technology development policiesand programs and in aligning them more clearly with private sector development and industrialcompetitiveness.

Among the different activities under the proiect the twinning partners brought value to the Indonesianinstitutions by the transfer of their superior R&D Management practices. The twinning relationships led toboth new avenues of technology and business develoDment as well as human resource development such asin training of leaders in R&D management. The project enabled some of the institutions in MoIT, LIPI andBPPT to come together as "Team Indonesia" for the first time. and offer services to each other- rather thanseeking them from abroad.

The process of business orientation of the R&D institutions not only moved forward as designed, but wentbevond the original expectations. It led to concrete actions of some institutions in setting up nf snin-offcompanies, each with their own way of doing it, e.g., the creation of Foundations and Holding Companiesas ltAfe.S-r.edia.. by L!P!; a.. t.e co.versi on of viable P" D ats zs D.fla-O Uy L.., . Ther

self-financing performance for all the supported institutions moved upwards in different ways. Forexample , 1JLUAS wiU. a Oelf-fr U1 ci Vatio f U M/u, achV evedr.. u.I.bUe LWV jJUpUsII basvu or, oveuI

earnings among all the MorT institutions, thus enabling MorT to contemplate the upgrading of BISm fromEchelon III to Echelon i . RDCAC's income from industry increased from 24% to 84% during theproject. BPPT's overall income increased from Rupiah 10 billion to Rupiah 15 billion in three consecutiveyears, whereas RDCAC's self-financing jumped from 16% to 43% in the last year of the project.

The Institutional Strengthening Component of the project completed a study on the formulation ofIndustrial Master Plan and Policy which led to the concept of a high powered commission named "NationalCompetitiveness Council" headed by the Coordinating Minister for Economic Affairs.. The Council,composed of high govemment officials, meets periodically to discuss current economic issues and events.The study also led to the formulation of a National Industrial Strategy where the discussions on "clusters"concept prompted the govemment to revisit its current industrial laws and the consideration for possiblechanges of these laws which would facilitate the implementation of the strategy.

Throughout project implementation, the Bank team complemented efforts of the research institutes inworking towards an environment where they can pursue freely their research objectives without beingrestrained by budgetary concerns. The Bank team exerted much effort in discussions with the Ministry ofFinance on the implications of Law 20 to the financial well-being of the research institutes. While it hasnot been successful so far, it has resulted in some progress in raising awareness to the problem among somekey government officials. The International Workshop on Science and Technology, funded by the Bankand which took place on June 17-18, 2002, has included this topic in its agenda. The objectives of theworkshop is to further raise the awareness of high government officials on the imnortance of Swadana

status for all the R&D institutes, to showcase the successful international experience in this area andhopefully be able to convince MOF and other high government officials to support the budgetary flexibilityfor the R&D institutes in Indonesia. The government is currently preparing a "Green Paper" which willtake note of the discussions and decisions from the workshop. The paper will serve as the policy guidancefor future R&D development. The deeper objective of this workshop is to lead to the MOF's action torevise Law 20 to allow budgetary flexibility of the R&D institutes. Since the workshop itself (along withits objective) is outside the scope of this project, the outcome of this workshop has not been included in theassessment of the project.

Overall, the project served the intended purpose of having a demonstration effect, influencing the overalltechnology policy and expenditure in the direction of commercialization, accountability and transparency.Moreover, the project facilitated the process of transfonnation by establishing restructuring incentives andprocedures among the R&D institutes.

I/ Perjans are government institutions positioned to become financially independent in the long Irun but are still subject locertain adntinistrative regulations in the short rin.2/ Research institutions under MolT are classified according to their scope of research activities and self-linancing ratio;Echelon Iii incitides those institulions at the regional level whine Echelon ff at the nationat level. Thne autter classificationdenotes a mor e prestigious position.3/ Swadana Li an exemption granted to some research institutes or universities fi-om comDlving with government hudgetai-vr estrictions which apply to most other research institutes.

Sustainability

The project components that appear to be sustaining themselves include: (a) continuation of the PolicyAdvisory Unit (PAU) as a think tank for GOI; (b) continuation of a DAPATI-like program by BPPIP(although on a smaller scale); (c) continuation of some of the partnerships with foreign twinning partners;and (d) the process of business orientation of the institutions leading to spin-off companies or theconversion of viable R&D units into Perjans. However, the issue of long term sustainability needs a carefulconsideration at the highest level in GOI. For example, maintenance of a modem technology infrastructureestablished in LIPI (Component E) will require committed human and financial resources. In general, inthe long run. the laboratories, outreach centers or other services that have been offered free by thegovernment agencies will have to become self-sustaining.

Several skills that were acquired by the Indonesian institutions either by twinning arrangements, or by theacauisition of software and hardware should be made available for the benefit of other institutions(including universities) which need these skills, by means' of "skills spread" program in the post-ITDPnhase.

Tn general; the sustainabilitv of many of the nroiect activities seems feasible. However, the issues ofsocialization, institutionalization and implementation have to be addressed, not in isolation, but by linkingthem in a holiitin manner to vario-us asnects such as ecnnTnumic incentives, institutional recLimen educattion

and labor.

Lessons Learned

1. In order that research institutions move away from being wholly government-funded and developself-financing capacity, the government should provide strong incentives for external revenue generatingactivities and allow the institutions to retain and accumulate their earnings, and to be able to use theseearnings with greater flexibility and freedom.

2. The Indonesian institutions are capable of providing service to each other, rather than seeking itfrom abroad. The "Team Indonesia" spirit developed during the project among the institutions in MoIT.LIPI & BPPT should be spread to universities with suitable incentive schemes. The experience from thisnrmiect with institutions learning from each other nroved to be very useful and ended with g,ood results.This lesson could be useful for future projects in the project design stage, to explore the possibility oftwinning arnonnr the instititions within the Gni mtrvy rather than always seeking help from abroad-

3. The Borrower shoufld consider additional capacitv bhuilding for imnplementing agencies nn theBank's procurement procedures. This is especially important for new agencies implementing Bank projectsfor the first time fhirincy the nrmient imnlementntion nerind bnrrowetr exnerienned nrnmiirement nrmhle-msdue to their lack of understanding the Bank's Procurement Guidelines. It is important therefore to provideimn,lpm,ntinr agencies the necessary., training un fmnt rather thqn when nrnhlems are detected dlringimplementation. By providing the training up front will also avoid possible delays in projectimpermentation ;nd the case on rmsnroc-remen.t.

4. The support required for SMITh is not in technnology anrd softw.are alone. Capita' financing to theSMIs to enable them to fully utilize what they have acquired in the Project (e.g. technology, managementa..A m,rlke,;ng skils, is .1-o vey, ;m-nnrtant and uet seer.ed lanli-n tn many nf tlw SIsATc -vhih ha-e

benefited under the project. Providing easier access to capital and technology for SMIs is a particularlyUItj.fJULLt facf.o Avr the; busit,,.e groth. Ln this r tn, t-jec t

s.-.oJLd ve beenhn..ted n a

broader strategy of assistance to SMIs.

| rJcID P037 IArjc.^' DAA2QQ e: ID-IND'Ld TE~iKc!CAf.JLG DEV~X

Team Leader: Lily Uy Hale !TL Unit: EASPS

JcR Type: Core ICR IReport Date: june 27, 2002

1. Project Data7M"me: ID-IND'L TECHLNOLOGY DEV L,rC!T,.N1,mber: CPL=39720;

SCL-3972A;SCPD-3972S

Country/Department: INDONESIA Region: East Asia and PacificRegion

Sector/subsector: DB - Business Environment

KEY DATESOriginal Revised/Actual

PCD: 05/01/1994 Effective: 03/18/1996Appraisal: 04/15/1995 MTR: 11/01/1998 10/15/1999Approval: 12/21/1995 Closing: 12/31/2001 12/31/2001

Borrower/lmplementing Agency: GOI/Ministry of Industry & Trade; GOI/lndonesian Institute of Sciences;GOI/Agency for the Assessment & Application of Technology

Other Partners:

STAFF Current At AppraisalVice President: Jemal-ud-din Kassum Russell CheethamnCountiy Manager: Mark Baird Marianne HaugSector Manager: Amanda S. Carlier Peter R. SchererTeam Leader at ICR: Lily Uy Hale Darius MansICR Primary Author: Lily Uy Hale; Sati Achath; Dr.

R.A. Mashelkar

2. Principal Performance Ratings(IS-U.nh1v tf>wtn.y, 1S=Shisa tort, t=tna T=T T6fhrtnrv MtT-=-TiThly T M ,T I=Ti.I, Th-T.1ITT=Tik-, FTTI TN=WH.hly

Unlikely, HU=Highly Unsatisfactory H=High, SU=Substantial, M=Modest, N=Negligible)

Outcome S

Ststainability L

Institntional Development Impact: M

B a n.k Pe'- .. an c e: S

Borrower Performance: S

QAG (if available) ICRQuality at Entry:

Project at Risk at Any Time: No

3. Assessment of Development Objective and Design, and of Quality at Entry

3. 1 Original Objective:

Th. nhiective nf thA TnJ,,cfr;a1 To,h,nln-, fa-,,pln-pont Project (TTnP) ,tas '.o ern,.o fhe

competitiveness of Indonesian industry, particularly the small and medium-sized industries (SMIs) by: (a)p.ro,v,id,ing publd., a nd prifva.he '.ecunrology support, ser.,iMes to'.o., (Sb)s f.bi. i -- r.g -..- ae t.opu..c ^.U

private technology service providers through a DAPATI Program; (c) strengthening and making more,nL %Lvi.avn. UJ tnn LtanJttJ au pnU lIFtllVU- Ut - &Y J mIjJLV- V USa, AVL,J,u, aUVII VA.U

coordination of industrial technology policies.

The objective was consistent with the Bank's 1995 Country Assistance Strategy (CAS), which emphasizedUIC UUjUI LaUt.,v, UI 1upJvvIUrVUI n UiUUv14a a %IUpiLuuVvLIba diLU U1S IIGU ivi a IUIL LUWd.Umb d

productivity-based economy and the complementary private provision of supporting infrastructure services.Lne project addressed each of t'hese areas througlgh its sULppoL for rmiore effecti-ve provision of technologyservices. In particular, the package of policy reforms, institutional strengthening and capacity buildingsupported under the project was expected to riake a contibution LU enUnce hte coUlCmpeiitiveness oIIndonesia's SMI manufacturing sector, the target group of the project. The experience gained under theproject was expecteu to proviuc a uselul ipul into uIdonesia s Lthnaology policies and programs.

i ne project was responsive to uie needs of ie Borrower. it was designed to improve mne productivity m metechnology area so as to maximize the supply-side response to the needs of the SMIs. Specifically, theproject addressed two important market failures in the area of technology-services. First, there was thedemand-side failure: many Indonesian SMI managers had underestimated the need for technologyimprovements and were reluctant to spend scarce funds on new technoiogies. Second, tnere was asupply-side failure: if left to the market, technology services would be underprovided on account of theirpublic good nature. Tne project addressed these market failures by initially lowering the cost, throughsubsidies, of both public and private technology services. In particular, DAPATI Program (TechnologyServices Matching Grant Scheme) was a response to the demand-side failure: its declining schedule ofsubsidies for technology services was specifically designed to encourage the firms to test the waters andmove up the technology leaming curve. This initial subsidy was expected to yield substantial fmancialreturns, thereby convincing the SMIs of the worthiness of technology investment in imDroving the chance oftheir long-term sustainability.

The project envisioned several benefits to indonesia, which included: (a) increased economic and financialreturns for SMls through reduced product defects, faster manufacturing turnaround times, lowerinventories, higher productivity, new product designs; and as a result of this, increased domestic andexport sales; and (b) substantial catalytic effect on other public sector investments in the provision oftechnology services. Given its focus on improving the incentives and management practices of theparticipating public technology support institutions (PTSIs), the project was expected to have ademonstration effect, which would influence the overall government technology policy and expenditure inthe direction of commercialization, accountability and transparency.

The project was also expected to have a positive fiscal impact, since it would enable the government todeliver the same level of technology services with a lower budget. First, the expected net impact of theproject on the private provision of technology services was expected to be positive; and second, theself-financing ratio of the PTSIs was expected to increase as a result of the project.

The project enabled the GOI to take advantage of the lessons from the Bank-supported technology projects

- 2 -

thrnnn'h-n *1-t u,nrlA Tt PrnohlaA flflT tn *oL- 5 I -hr rtnhn *r- , nnennntroala t. flfll. It enabled GOI to 1'ke aamore p ai star.ce in focusirg 101 t.oloJy

development policies and programs and in aligning them more clearly with private sector developmentneeds5 ar.d industni,al cor,pe,rtitves. 'The Bank also played a tcey role_in n _bls- -aico..nn policy ar.

institutional framework for industrial technology development that was being supported by other financiers,unld. the~ Asi,ar. felo.a,i..am,,t Bak

i Rn project 1.-zed ar.d took i,t.o accoukLt two risk facto10 wi, c WLCh coUUld afll,cL te pILUJect

implementation:

The first risk factor was the slow pace of reorientation of the public technology infrastructureLUWaLd.U kVUill-leil a, andU L- -OL-inL1LVU managemeLItL, effeLVctiv I..JLUUiuLaLVUL UL r,-V1UnerLt poiLciVs for

industry, science & technology and the development of competent private service providers. Likewise, the.lvu secLor Loo co-uuud be slow uiaccessing uu utilizuirg 'uhe new sevices bezing offered. To deal witu uleserisks, the project preparation was extended to: (i) help build consensus within the government on the needto achieve strategic reonentation of public support for technology development, including drawing on tnebest practices from other countries' experiences; (b) develop a new policy framework that would driverTI s to commercialize their research and development and to deemanu ior their greater accountability inachieving measurable results, increased transparency and better management of operations; (c) provide theparticipating PTSIs with an opportunity to develop sound business plans for their commercialization; and(d) obtain the participation of private industry and professional consulting associations in project designand inplementation.

The second risk was the project's sustainability, since DAPATI was only a piiot scheme and therewas the risk that due to budgetary reasons govemment would not continue to fund it after projectcompletion. This risk was addressed by a pianned evaluation of DAPATI which would provide thegovernment with a very good informnation base to decide the future of the program; and also by the Bankreviewing with the government throughout the project implementation period the progress of DAPA TI andplan for its extension after the closing of the project.

At the onset of project formulation, it was recognized that all the benefits of the activities supported underthe project may not be easily quantifiable. The research centers were given numerous twinningarrangements with research institutions from countries with different levels of technology attainments andcultures. The project was also very demanding on the Borrower, since it involved networking of manyministries and implementing agencies, which had no experience in undertaking Bank projects. Therefore,developing a coordinated strategy among them was a daunting task. In addition, a large arnount ofprocurement activities required substantial knowledge of the Bank's procurement guidelines. Throughoutthe project implementation period, the Bank conducted several training courses for the implementingagencies' staff which minirmized the problem of lacking procurement skills of these agencies. In retrospect,if some of these training workshops were conducted up front as project preparation, the additional intensivetraining workshops could have helped avoid some implementation delays.

3.2 Revised Objective:

Even though the project experienced restructuring three times, these were debt portfolio restructuringinstituted by GOI across U 'uiboard -which UUlnacU all IoDIU loais, uin uudirg uie ITDP~. Tisii udui i2fru fo

some restructuring of projects which aimed at ameliorating problems encountered during projectimplementation. Tne inpact of these series oi restructuring on ITDP was a scaling down of activities fromdifferent project components, resulting in a cumulative loan reduction of US$8.5 million (from US$47million to US$38.5 million). Notwithstanding the scaiing down of activities, tne objective of the

- 3 -

components were unichhanged since he canceuiauun ui tie acu-viues was done uiruugn a careiui ana

thorough review of the Bank team with GOI project teams, bearing in mind that any cancellation would notaffect the achievement of project objectives. For example, training abroad for some R&D center staff andthe subscription of journals and books, which during project preparation was deemed important, wascanceled as much of this knowledge/information from joumals and books is now readily accessible freethrough intemet services.

Another scaled down activity was the magnitude of coverage of the DAPATI Program. The objective ofthe program was to introduce technology services to SMNs through initiai subsidy. The originai budgetallocation for the program aimed to reach 600 SMIs was unrealistic as it was under the assumption thatonce the project is approved, the program could be rolled out smoothly without issue. in reaiity, theprogram took off to a slow start and also took longer than expected due to the learning curve experiencedby the DAPATI Management Contractor in implementing the DAPAII Manual in accordance with theBank's contract and procurement guidelines. The 1997 economic crisis and the domestic politicaldisturbances in the late '90s exacerbated the slow pace of the program.

The portfolio restructuring therefore provided an opportunity to reassess the implementation of theDAPATI Program. While the original objective of reaching out to SMIs to encourage their use oftechnology services remained unchanged, the reduction of the target number of SMIs provided a morerealistic indicator and also resulted in a loan saving through reduction of DAPATI grant allocation as wellas the contract amount of the DAPATI Management Contractor.

Finally, the 3rd phase of the twinning arrangement between BPPT and its partner Battelle MemorialInstitute (BMI) was cancelled and the task of its implementation was undertaken by a local consultant andthe BPPT staff who have benefited from the first two phases of the twinning arrangement. The partialcancellation of the contract also resulted in loan savings through the restructuring process.

In sum, the portfolio restructuring provided the project an opportunity to reassess the needs of the projectactivities at different stages of implementation, leading to a loan reduction/saving of $8.5 million incumulative terms, yet without compromising the achievement of project objectives.

3.3 Original Components:

The project design was well prepared and the components were related to achieving the objectives stated in3. 1 during project preparation stage. These objectuves were planned to be achieved througn the followingkey strategies:

Provide Indonesia's public technology support institutions incentives to become more demanddriven and commercialiy oriented, and establish targets for them to become increasingly seif-financing.

Reduce government controls to encourage the institutions to become more agile in responding tothe technological needs of industrial finns, while also increasing the institutions' accountability for results.

Implement internal management reforms at the institutions to ensure that they become moreprofessionally managed and results onented.

Component I. Improvement of Metrology, Standards, Testing and Quality (MSTQ) Services

The MSTQ component comprised of: i) Providing quality awareness among SMIs; ii) Accreditation ofquality systems of SMIs;- iii) Establishing outreach centers for assisting SMIs; iv) Upgradinglaboratories under MoIT to improve its services to SMIs; v) Upgrading and assessing the National

-4 -

ivivuiugy l V1ILV1 VI V1 -and.-VI.) UipLIadiLUg ar.d assing cl1IUIa.UII labUIdLUIriS -IuIer tue INaional

Calibration System (JKN which aimed to provide better services to the R&D centers.

Items (i)-(iv) were handled by the Ministry of Industry and Trade's (MoIT) Center for IndustrialSinuarus OPUSTAN) anu iLems (v)-(vi) were handled by the National Metrology Center tKuVi - LiPI).The former was done through a Master Plan by providing technical assistance (TA) and training to improvethe dissemination of setadards to industry, especially SMivs; and impiementing programs for improvedtesting and calibration services and national certification of product quality and quality managementpractices, including iSO 9000 certification for industries.

The project provided TA to: (i) KivM through a twinning arrangement with a leading intemationalmetrology and calibration laboratory for building its management and institutional capacity, includingimprovement of its tecniucai proficiency; (ii) ruSTAN with an intemationally reputed accreditedlaboratory to improve the performance and institutional capacity of five testing laboratories of MoIT withthe objective to gain intemational recognition and accreditation. In addition, it covered training of staff ineight MoIT R&D institutes as MSTQ extension agents; developed cooperation with the IndonesianChamnber of Commerce and Industry (KAI11-N) in conducting programs to promote quaiity awareness andimplementation in industry; and initiated ISO 9000 certification of 40 SMls as a pilot demonstration to therest of the industry and as a leaming-by-doing training program for their staff.

Component II. Diffusion qf Good Technology Practices (DAPA Ti)

To improve the provision of technology upgrading services to SMIs, the project supported the creation of aTechnology Services Matching Grant Scheme (DAPATI). DAPATI was demand-responsive and promotedprivate sector delivery of technology services, while avoiding the creation of new govenmment agencies. Thefunding was done in the form of grants to cover part of the costs of different technology services At projectdesign, the Program was set to target 600 SMIs of all subsectors. Subsequent events such as the East AsiaCrisis, and the political turmoil in Indonesia in the late '90s, however, led to drastic reductions of theprogram target to a more realistic level of 130 contracts.

Component III. Improvement of the Productivity of Public R&D

(a) Improved management at BPPT and LIPI. The project supported improvements in the intemalmanagement systems of BPPT and LIPI through introduction of measures to improve their autonomy andaccountability, and to provide incentives for their effective technology support services. This was donethrough twinning arrangements with internationally reputed contract research organizations. The focuswas on strengthening their capabilities in areas such as business development, contract administration, andintellectual property management. The objective of this component was aimed at providing better qualityservices to SMIs, thus increasing their competitiveness, through improved management of their serviceproviders, BPPT and LIPI.

(b) Restructuring selected laboratories. In general, the R&D capacity of SMIs is limited, whereas thepublicly funded R&D institutions (with 80% of Indonesian S&T force) remain insensitive to the needs ofSMIs. To encourage these institutions to redirect their focus from pure research towards increasedpartnership with SMIs, three laboratories under LIPI and MOIT were selected for comprehensiverestructuring which included improving their R&D capability and their management skills. The objectiveof these efforts was to encourage the R&D institutions to provide better services to SMIs. The Bank'ssimilar projects elsewhere (e.g. India) have shown that such a restructuring of institutions helped SMIs inreceiving better technical services. Restructuring of R&D institutions is thus an integral part of any

- 5 -

technology linked project of the Bank.

Component IV Improvement of Industrial and Technology Policy Formulation.

This small component, which consisted of TA to the Science and Technology Policy Unit of the Agency forthe Assessment and Application of Technology (BPPT), was essentially meant to strengthen the ability tomonitor and evaluate the impact of ongoing technology programs. It also aimed to formulateresults-oriented technology policies and more effective mechanisms for improving Indonesia's industrialcompetitiveness through technology upgrading. In addition, the project also provided TA to the PolicyAdvisory Unit (PAU) of the Agency for Industrial R&D (BPPIP) under the MoIT to design collaborativeprograms with the private sector for strengthening industrial competitiveness and to prepare the next phasesof the deregulation process.

3.4 Revised Componenits:

Not Applicable

3.5 Quality at Entiy:

Not applicable.

4. Achievement of Objective and Outputs

4. i Outcomelachievement of objective:

Th ~ cn,nnaaA ni . rnnt b. -h0 n,,4- o^onnc f QT%Air h1, -AA.-a fa-knno-The proJe.,cceeded iFr.JLi 5 -I.- -t,h.e .i a..a.OsJsL J kfJ*'J 5 -V-- tec.'ica! ssifce using

various means which resulted with the following: (a) quality awareness seminars were conducted in 9 areasUi. ILdonesiza a. attendiAanceo 3160 SJ.J cor L .s..flfpanie i j 0) qrLt aVl documentation ..jughin-house training were provided to 672 SMI companies; (c) quality systems within enterprises were

imp Ar.e, (d)t AA SQAArT. A--, - A -- A.e -1-selcte for ISO onn0 cerufication;- (e) provide helirUUJ~LL JI t%LLVJ,~ I,U *V lvLJLUO W~k 144%41&&LiS auL SUUO1 .Vt V &'J VJJ'A LaafJi JLU VLUI~U AII LlU

internal quality audit; (f) provided training and implementing management review; and (g) assessment. Italso crecateud a posi.v i...pac on LSX A'T_ in t...- s4 of -.-i inc.-sed -- A-u4-t 1owe; proucton ejetio41~UIA~4VU 4~U~LLLVV 1LL4ta% USA OIMVUL Ui L~.LJ.IM VA US~A1LAL A"(S4a0U jALAUVU%,V1LY, LVWVL Pt'VUU%L.LIUA 1-~JV%."VUU

rates, and improved management, thereby enhancing their competitiveness.

Even though only 27 out of the targeted 40 SMls under the PUSTAN sub-component obtained ISO 9000ceni[Ui uon, hle rIeruuug 13 aivius ha-V- pabssu xi-.WU. LA.l-i Loia for VV ISO 90 U dia reduy LU ouLain Uie

certification once funding becomes available. These SMls have acquired the skills needed for ISO 9000but short of having a certuficaW o show- tWheir qUaiuiLauUuiL ih [i iIg-UHr ne UL Ulf: aiVUS d0 not nave

funding to pay for the certification is not very convincing. It is possible that the SMIs were not totallyconvinced by the merits for obtaining iSO 9000, and thus hesitated 'to spend the rnoney, especially underthe tight economic situation.

Under the DAPATI Program, most of the SMIs improved their productivity and increased their salesrevenue (the increase ranged from 30% to more than 100%). As the Program became well-known overtime,a growing number of SMTls (from 27 SMls in 1997, increasing to 241 in 1998 but slowed down quite a bitin 1999 to 74 and the pace picked up again in 2000, reaching i97 Siflvs) had approached the ManagementContractor (MC) for assistance and expressed interest to participate in the Program. Total grant valueamounted to Rupiah 4,520,626,275 (60%) with the share of the SMIs reaching Rupiah 3,05i,393,858

(40%).

(ii) Twinning Arrangements:

As part of the program to improve R&D centers' skills in providing services to SMIs, the project succeededin establishing cooperation between the researchers from the Indonesian R&D institutions and their foreigntwinning partners. The twinning partners brought value to the Indonesian institutions by transferring theirsuperior R&D management practices to these instititions. They contributed also to reduce the costs ofprocurement of equipment, and modules to suit the local environment and context More specifically, thetwinning relationship between LIPI and CSIRO as well as LIPI and KIST led to both new technology andbusiness development as well as human resource development

Under the Management System Strengthening (MSS) component, LIPI's twinning partner CSIRO(Australia) provided help in setting up a Project Management Resource Center (PMRC) and a TechnicalService Office (TSO). The PMRC specifically involved the aspects of Management Levels I/II changemanagement program, a leadership development program' (P2K) and a staff varticipation and supportprogram. The special workshops on team building and research planning, financial accountability, researchpriority framework and planning and evaluation framework helped in terms of fulfilling the basicobjectives of the change management. The P2K programs focused on management of creativity,development of communication skills, and scenario planning capability.

TSO established and made onerational in prioritv areas the core functions for the management oftechnology, including business development, contract administration, intellectual property management andpublic affairs and communication. The Center for Innovation. established in June 2001, not only assistedLIPI in becoming a better contract research provider, but also in assessing S&T capability and businesscollaboration within Indonesia and overseas such as China, Australia. Korea. and Janan

Likew.ise, the twinning arrangement with TNO (Netherlands) had significant impact in enhannina the

technical, research and development and managerial capacity of BISm staff and in making the institutemnore client focused and qervire nriented, nnd increasing its ahilitv to offer hetter uinlitv servincs to

industry which eventually led to increased business and income, thus achieving a higher self-financing ratiorennhincr 600/.at nmiere rcInoincr

Tinder the Mansgement & Business Integrntion System (MBIS) compnnnent BPPT had a stWinninuarrangement with Battelle Memorial Institute (BMI), USA. Overall, there was a marked improvement inhiiildina rtierhnl filn-tinnc in thp .rent of h dcinPcg develnnment re%nftrnt m innwopmpnt nilhlir rplntinne snrl

communications, and intellectual property management. The creation of centralized business developmentnrornnti7n1'inn prpntinn of mnrtaetina tnHtnhaeo nas waell na Atataev on BPPT nanIaipitie A-A- ni of

standard contracts and contract tracking systems, improved public relations, development of inventorsguides, nnhnnrd flmnn of -atonts _sv! as creation of nfiicarnro ar.d rognitionrnnran,. wereclear gains from the ITDP intervention. What was missing in this twinning arrangement was the latter parton the gud,ance - ro-,Aded to BPPI on the A-nlementatin an.d policy directins. BPPT was not con.vincedthat Battelle could handle the task and undertook a technical review of its performance. The outcome of the

the cancellation of h.e third phase implementation by Battelle. Instead, BPPT used a localconsultant to continue the Phase 3 implementation which achieved the objective of this phase.

In general, the Indonesian R&D institutions acknowledged the benefits brought by their twinning partners.Ther.z.-b - .t.. p-vr.nf +U-,e, p.o.^-.c can -be -- evide-ce tho¢ e incree in +kheir --vreanA 1kV LItOCLLr'.U ULjLqU a V VIIPU JL UE,U p~IIVILL~UUI%A 'u U%, ~V IG1%Lf.vG ULvtu JU it 111CUIMSC 11 UKII, Liiwi11i, ailu

-7 -

self-financing ratio during the project implementation. Tnese improvements in turn showem the wiiingnessof 'SMIs to use the technology service providers which links directly to the project objective of improvingthe competitiveness of the SMivs.

(iii) Strengthening of public technology support institutions and iteir responsiveness to the needs ofSMIs:

KIM-LIPI: The project was instrumental in increasing the capacity of KIM-LIPI to a substantial extent.For example, its efficiency and capacity were enhanced in terms of shortening of calibration time,achievement of higher accuracy, and avoidance of the need of calibration by foreign laboratories as a resultof internal capacity building. While its own standing went up with its ability to participate in theinter-laboratory comparison programs in the Asia Pacific region, it was also able to provide better supportto indonesian industry and trade more effectively.

Good progress was also made in the area of intellectual Property Management, especially the training inthe drafting of patents by using the personnel from the Indonesian patent office and the CSIRO patentattorney.

BPPT: The project helped in changing fundamentally the orientation of BPPT towards industrial research.BPPT became more responsive to the needs of the SMIs rather than merely performing as a governmentresearch institute serving solely the needs of the government. BPPT also networked with other Indonesianresearch institutes. For instance, LIPI and MolT for the first time collaborated in doing joint researchprojects, and providing training from one institute to the other. Likewise, BISm was able to seek servicesfrom BPPT rather than from a foreign consultant.

As BPPT's main instrument for its business development and management of its project technology serviceactivities, the Technology Business Development Office (TBDO) was formally established in August1998. A center for Technology Services and Information (PUSYANTIS) was created, which replaced thefunctions of both TBDO and Yantek Forum. The main task of PUSYANTIS will be to serve all theBPPT work units and also manage BPPT's information systems and secure the sustainability of ITDP andBPPT effort to become an effective contract research organization. The impact is already visible as judgedby income rising from Rupiah 10 million (1999) to Rupiah 15 million (2001), patents rising from 0(1998) to 6 (2001), and self-financing ratio changing from 5% (1998) to 13% (2001).

(iv) Self-financing Performance:

The self-financing performance for all the supported institutions showed improvements in different ways.For example, BISm with a self-fmancing ratio of 60% in 2001. achieved number two nosition among allthe MoIT institutions, thus enabling MoIT to contemplate the upgrading of BISm from Echelon I toE.heInn n RTnCAC's incrnme from industrt increased from 24% to R4% diuringr the nrmjert BPPT'soverall income increased from Rupiah 10 billion to Rupiah 15 billion in three consecutive years, whereasRTC.AC's self-financing juimped frnm 160/% tn 43% in the last year

The Instit.utional Strengthering Corm.ponmenrt of the prject completed a study on the form.ulation ofIndustrial Master Plan and Policy which led to the concept of a high powered commission named 'NationalCor.ptitivr.es Colln hne1add kb the (nrenntinc.g Unictpr for Prnrnnmir Affairs The or.,

-8-

cor.posedA of hig,h gonxn,remnt n#nulcas meetsr ner.dinll.r tn,o discus c.rnt eotnrr.omic isuesar.. d eve.ts.

The study also led to the formulation of a National Industrial Strategy where the discussions on "clusters"cor.cept promp'.edLS ~ ' g .aflnflt.4l I s- h c r.er.t ,ht.a.4l laws ar.d f-he corsieA--n+;- for possible

changes of these laws which would facilitate the implementation of the strategy. These actions, the meeting0of +1IS cou.ci1 and L th V- ft o fl th tfl erAA-.t* 4.1 La. *OVV lav .a;.ln progress --sultr.g; +L-. stuAd

supported by the project during its lifetime. It prompted GOI to actually take actions, as opposed topre,viu prctc -ir. ;_+-'hePO D cener c hich - sl- y put1 -u. coplte studies_ onA- shelves w-i-hout 9-rvherjJl',. LJM. " pL.-tR a. u.1, ANtXZD V111a -VU~.i UaUaJJ U. FUa LLJLV,%VJI4 1ULVU OLUUAVO VJU ILAt.L V ~ YIUIUUL IWUIWL

actions.

The workshop for the Capacity Enhancement for Technology Policy Analysis and Managementk% I t.IM) ylielUd saisfac.Lo1y L;WLU WIUi LWU %.-vvoIoncVr UvVulopulmiu, vIL. uIV urinral enLuncia4llI Uo UCe

National Technology Policy for the year 2000-2004; and the integration of efforts to create the IndonesianOA' . ...... 2..L ... I... L. ""fTV"' _J - -U .11 .33- i

a&T iLLUiUU15 WURA.1i are now Ubing usdu uie BPU P r anD-i hich wiln evv-nLualy spreau Lo ani I¶LJ

institutions.

The project was successful in achieving the original objectives despite substantial reduction in the loanamount. Although ute onginmal ojecuv was osuy to ornent te inmsuLuons towarus oifenng services toSMls, the project was instrumental in the setting up of spin-off companies by some institutions. Theexamples are, the creation of Foundations and Holding Companies as intermediaries by LIPI; and theconversion of viable R&D units as Perjans by BPPT. This is a far cry from the early phase of the ITDP,when institutions were struggling to come to terns with the idea of even networkicng witn businessenterprises. In addition, LIPI moved in newer areas of research such as new bioactive therapeutics based ontraditionally used medicinal plants. Tnis technology has been already successfully commercialized and isbringing income to LIPI.

1/ Levels M/lT: classification of management level within LIPI2i YaI tek rul:um. I nferfl fUjinal mu-nagemwun System in BPP

4.2 Outputs b, cornnponnents:

Component I: MSTQ Services

(i) Programs of PUSTAN-MoIT:

All activities under the MSTQ component for PUSTAN were completed:

(a) Technical Assistance to promote quality awareness: 3163 SMIs participated and benefitedfrom the quality awareness seminar, 672 SMIs were trained in documentation of the qualitysystem of ISO 9000 and 27 SMIs were certified into the ISO 9000. This activity was completedearlier than most other components and the budget allocated was completely disbursed at least 12months before proiect closing.

(b) Technical assistance for setting up outreach centers: This sub-component was successfullycompleted in November 1999. Eight outreach centers were established in eight regions, 64extension agents were trained in extension services and 1768 SMIs were assisted by the extensionagents.

(c) Upgrading laboratory management and conducting inter-comparison tests: This part of theprogram comprised the provision of technical assistance to five testing laboratories of MOrT. All

-9-

UAC -Vi-.d elen Vo Le. j.-o&I. w- -eS OUSA-.su ly coArA.lte by MIIA- 2000.

i P A UtOgr LI, of Y A-An riL A.

fl,L VILALVIU3 AUVLLL U.JJ LAJLLLJUV"L.L W-6iL, DLLJa aA~AiL 1U& LLpI~LLL.u.

ka) T wuin,g AzfUflgentwicri. CS xuOL, LausLia' prumary IAIVAru y ceneLV1, adIU A'ilvi-LIrl ihud

twinned together. All tasks covered by the contract between these two institutions were completed.

(b) Procurement of equipment for KIM-LIPI: The procurement of laboratory equipment was doneuirough two ICBs, Une simgle suourc p Llase Und. one :siteanal sV ppig.

Component II. DAPATI

148 contracts witn SisnU were signed, of wuich 14' contracis were Iuly iImplemented and accepteby DAPATI. There were 86 Service Providers (SPs) assisting these SMIs. The total grant valuewas US$5i6,450.9i (see earlier paragraphs for Rupiah value). Technology seminars wereconducted in Jakarta, Bandung, Semarang and Medan in December 2001, and these were attendedby 100 participants.

in addition, training sessions were conducted on: (a) Kiln Drying of Sawn Tiimber for Export andWooden Fumiture Product Quality Control for Export; (b) Waste treatment Management; (c) Fishand Peanuts Processing; (d) Textile Garment Design and Finishing Techniques; (e) EngineeringDesign of Agricultural Machinery; and (f) Metal and Non-Metal Foundry Technique for SMIs.All these trainings were conducted for SMis in improving their skills in different areas which areexpected to improve their competitiveness.

Component III. Improvement of the Productivity of Public R&D

LIPI Programs:

New Business Development Guidelines were formulated, which include research planning andpriority setting, communication strategies, account management and issues dealing with standardcontracts and legal aspects. Further initiatives included the formation of the LIPI Foundation andthe formation of the LIPI holding company. These are expected to provide the necessaryfacilitating mechanisms for technology transfer, creation of spin-off companies, and easy flow ofthe proceeds of the profits of the companies back to LIPI.

The RDMS deliverables included Financial Information Systems (FBMS-OBMS), HumanResource Information Systems (HRMS), Project Process Management Systems (PPMS), creatingWAN and LAN. However, LIPI-wide implementation of the systems was not very successful dueto reluctance of some managers to switch over from manual system to computer based system.

The RDCAC-IRDCI (MoIT) twinning program led to an initial partnership between the twoinstitutions with the joint research projects which successfully produced results to a new product.While the twinning objective has been fulfilled, their further cooperation could have led to furtherimproving the products, with potential use in the leather and cosmetic industries. However, thispartnership did not sustain. They did not continue the partnership beyond the project. Thediscontinued relationship could probably be due to personality issues between the two institutions.

- 10 -

On the other hand, IRDCI's partnership with NCL (India) has continued beyond the project.

The spin-off results of the work done at RDCAC in terms of their setting up companies infermentation technology moved from the pilot plant stage to the commercial scale plant at projectclosure. With a turnover of over one billion Rupiah and an assured royalty of 3%, there will be asignificant revenue stream for RDCAC.

The Twinning Program between RDCAC and KIST resulted in a joint business operation betweenRDCAC and a Korean company, KORTEK A pilot project is planned to comnence in July 2002.

MoIT Laboratories

(i) BISm : Semarang Institute

Procurement of laboratory equipment/office supplies. This part was fully executed with delivery,installation and training complete in all respects.

Training Activities. In addition to capacity building, this component offered BISm theopportunity to avail itself of numerous training programs both domestically and abroad. Theseprograms also exposed BISm to market opportunities abroad.

BISm rose to number one position based on overall earning among the thirteen other regionalinstitutions, and is ahead of several 'Balai Besar' institutions. It also benefited by new Internalnetworking with LIPI and BPPT such as obtaining training from LIPI and BPPT instead of seekingforeign assistance which led to some cost savings.

(ii) IRDCI (Institute for R & Dfor Chemical Industry)

Twinning with National Chemical Laboratory (India). The twinning with NCL helped IRDCI indeveloping better models for institution-industry partnerships and in better delivery and marketing.The decision by the senior management of IRDCI/NCL to continue the cooperation beyond ITDPgave IRDCI a sustained advantage.

Twinning with AFNOR (France). This twinning arrangement helped the packaging division ofIRDCI in improving the laboratory management practices, in creating improved testing services,and entering into new technology domains in packaging technology.

Laboratory Accreditation. IRDCI had an arrangement with NATA of Australia for the laboratoryaccreditation program. This program was meant to get international recognition for ERDCI inpackaging, chemical, microbiology, waste water, and calibration laboratories. The program wasnot completed at project closing due to a variety of reasons which included administrative delayand unsettling political environment which caused security problems for the NATA staff to visitIndonesia, since their time of visits coincided with domestic internal political disturbances.Although NATA proposed to visit Indonesia again at the beginning of 2002, unfortunately, theproject was closed in December 2001, prompting the activity ineligible for financing out the loan.IRDCI will continue pursuing other means to continue the objective of getting the internationalaccreditation of these laboratories. The other objective of getting these laboratories accreditednationally was performed hy KAN- the National Accreditatinn Committee of Tndnnesia.

Agency for the Assessment & Application of Technology (BPPT)

As a concrete step in establishing strong links between the laboratories in BPPT and the privatesector industry, BPPT had set up the process of preparing several of its laboratories to betransformed into business entities. BPPT conducted a feasibility study of spinning-off of some oftheir technical units into financially independent business entities. Based on the outcome of thestudy, BPPT has decided to convert IPTEKNET into Perjan. A proposal has been submitted toGO!, and npparently, it is still being considerec With the heln ofn lcnl consiIltant, BPPT carried

out the evaluation of the remaining 17 laboratories. A relative ranking was prepared on the basis ofthe SWOT nnnlv5sis Arnrdincolv the hiiqin,qs npnns fnr qPve-nI 1nhArntnries were heing nrennred

by consultants. These steps are in the right direction, and indicate the willingness of BPPT to-nder- a fiLndamentani transforma.tion to mee the challenge of th. fitiirp* TUnrler RrlM5q

component, the two activities managed by PMIU-BPPT were RDMS training and the computer.nd .- ,-nfp,--pimont Thp' were vtsrtMA in earlu .Mau 200! a-snd nninilftied in bJnvemrHr

2001.

Component IV. Improvement of Industrial and Technology Policy Formulation.

Institutional Strengthening (Mol):

The Policy Advisory Unit (PAU) under MoIT completed several studies including the study onIut.A-_+; flU1 -.OO -d D. lWf anl.A 'fl.DJ' 1;- AJSJ+*U tOA .-lA1 L AJ Cf*Lfh JLtlJtA- f D..f-t&tAOttAL fldUL&usu ., ase P.. d Poicy lecto... -sec.o - td _ d th--e Study on.su.-.5 -gothe Agro-based Industry. These studies were put into good use through implementation of therelco.-ll,ULLUeauivU uy U.v1 sMuUiS. A v1 eLX&Imple, La1L1 jF- *c .. . ... .l.. .... V.11.i

government officials meets periodically to discuss current economic issues and events whichso111MuLUS leu Lu avuou VI ILLUME1 o11i'y discUssioUs.

Iru-stitutiu-nal Strengthening-° PT

Inh Scieu(e UU a Tciiuilgy 1 dVIangeIUenL icULy 0l) lVI) groIuJ cm1Lp1lvUM a stLUUdy oL uilV

assessment of current technology policies. The most important output of this study was raising theawareness on tne i-mportance of systernatic technology poicy a-ssessruents wi iUL Br T.

1/ Perjan iS a government insUilhtion under BPPT positioned to become financially independent in the long run but is

still subject to certain administrative regulations in the short run.

4.3 Net Present Value/Economic rate of return:

Not applicable

4.4 Financial rate of return:

Not applicable

4.S Institutional development impact:

An important outcome of the project is hIe U-ansformraiUon of BPP aDrr ageUny Iuure mespuusi-ve t 'uhe

needs of the SMIs because it responds to specific demands from the SMIs rather than just undertalcingresearch for academic interesis. Tney are also more cooperative with other research institutes suca as those

- 12 -

inder ,IPI and MoITT As a res^lt of the intiatives ermrerging froM TDp, BPPT rndervent-restpJcturjrgas outlined in the Decree by the BPPT's Chairman on March 30, 2001 (No. 021/ KP/ KA/ I!I/ 2001). Inthe new structure, a Center far T, echnology InfLormnaticon and StandardiztiSeonices [PTTVAKTTrTQ istup to improve the implementation of tasks and functions of BPPT, particularly technology service to'n ,o triad^^ i.es.

Th e specia v,ork.b1n- on tea, r. bA ;Ifing and -lannnAn- .dted, Tne LII - -ograr.., he1leA the

change management. The P2K programs produced 47 managers, many of whom now hold Levels 1/11positions a.-A of L ,e,od to r t i-n leadershi positions. TSO established _- -- A

made operational in priority areas the core functions for the management of technology, businessA,oIn-..o.+ -A ----. +;ons.. T TbT I--.1- U ,4aro+- T.-- -+; ID-. T--.,..

o.. .a.e.. aL.d c.t .Q:-aL:ors. LIas, also 0 esa %,tablisheAd a C.erAAU VA ML, atILIon (Pl UsaL Ilovasil).

Furthermore, LIPI has initiated a Forum of Communication for Researchers.

MoIT has established a core team consisting of 15 persons for planning and formulating National IndustrialPoliLcy 0I LU.donsla. IL Liao also UUiIL up a goou WWvel uo %,VIUpe1L.Wny iL ULYVADV, dIV43 V1 Lto.lI11ULUry Ucu13.ra,

technology policy formulation, and technological innovation.

The twinning arrangements with various R&D institutions from different parts of the world facilitated:L.X...L. I ti o.r 0. % 1 ..e -Z .:.. -.... - 'ThI T TIlT '.. TTCV.stren;gul1eUn U1 .AX 111411=8VIIIVUL erCi4UVU VIs IIUUIX.011011 in,ULUUtion such 4a r.UVk-L.Ji, and BISD1OM.

Some of these partnerships such as IRDCI/NCL have become self-sustaining and will continue beyond theproject.

5. Major Factors A ffeting In..plermentation a-A Outco.e

5. / Factors outside the control *^/governnment or implementing agencv:

The East Asian crisis of 1997 had caused some inevitable delays and to some extent cancellation of someof the project activities (as discussed in the previous paragraphs). Apart from creating problems inbudgeting and counterpart funding, the crisis led to a period of inactivity within the country in varioussectors, which slowed down the progress in some components, such as DAPATI. It also created a newchallenge to the technology institutions, since their earlier projections on business and earnings were nolonger valid, and as a result, all the institutions had to reassess their performance indicators. Many SMIssuffered from the financial crisis, and research assumed a lower priority. Despite these events, the programmade satisfactory progress.

Owing to steep devaluation of Rupiah in the wake of the crisis, implementation of some components whichwere denominated in the then Rupiah amount was adversely affected. With the delay in projectimplementation, the amount of Rupiah was no longer commensurate with the activities that were to beundertaken subsequently in some areas. Implementing agencies with activities affected by the currencydepreciation consequently experienced implementation problems due to the budget shortage.

5.2 Factors generally subject to government cont-ol:

Swadana: The Ministry of Finance's Regulation of 1997 (known as DIKS) leading to Law 20 wasA.atIA.111.lJ cuUUWAr,otL OAdA a L.u0LAA.fLnVt. to Uth UiO-'.J jL IIJJO I sUAeU ItIA.U11UU5 1LV011

industry. The procedure involved anticipating the earnings a year in advance, depositing the revenues withUL.e 1V11LitU_Y, UdLirir.g LLLIt UbUdgeL r.eeds u.AIde; WILL IL it VWlI Ube OsJp.nL, CnZILA Os.g.itlin prio; pJJ.Ut 113Lui U tu.a1t-

any changes. In some cases, MOF had actually reduced the targets, that were set by the institutions, ar_iv th_t a c lea ._l- ag- . . A -- _ _ As:A..A+_A 1____ I L_A f_Al^I. T T,s... AIl 1110U.L W43t wa::Udiculy 0aU1aL LSL UAIVC U 1 U1 Ar l4UL 3U11-ULa.11 JJA%;IUD33vU Uy (.Jwl. IL waa 3 Utt UIOLd

- 13 -

tLh.e syi.,atiors wera .sipnI. .ne tbeir energies on fndmng ,l+a,-,t, ,vays to c:, rn,., t the,systr-,

something that was totally counter productive.

Due to the East Asian Crisis, GOI initiated portfolio restructuring three times during the imnplementationperiod for all Bank projects; n as a $8.5 r, ffr TTQUAS7 mili- to US$38.5 --

cancelled. In response to MOF's call for loan cancellation, the Bank conducted a thorough review witheach;.pemrdn agency on a-11 +U-. reriir. t-1 b^t e-d Ln -an _444ao.ce ed ef=ot sor.e of

v4l0 AAl.41'aA.el 5-J. Jl A .lfV.- -r L-O.N0 US VJL UU C-I.Ja A.iL 1,U CW.LU WIULLa conUcL,rteLU %I L , i U1 'I1, VJI

these tasks were cancelled. All cancellations were carefully reviewed to ensure that their withdrawal fromtLhe pkVJJ'-l not -Jl Q LJ.e ai oUe'Ve-ILMAf J1.JL UUJVWLLV k L1I03XLctiJ.L (seeIC CasC CdU'CCled

under the paragraph on "Revised Objectives").

1/ Swadana is an exemption granted to some research institutes of universities from complving with governmcni biidgelaryr.ctrictinnv whi.h nnniv in mn.ct nther r-cemr,Jh invtihitP.C

5.3 Factors gener ally subject to implementing agency control:

Procurement. Lack of understanding on the part of some implementing agencies on the Bank'sProcurement Guidelines in general, and the Procurement Plan for this project in particular, was a majorproblem which hindered project implementation. Many unnecessary steps were taken erroneously and therewere delays in procurement processing as a result of their weak procurement skills. For instance, ex-postreview of three completed DAPATI contracts showed that there were some problems in the process ofprocurement The selection of the Service Providers (SP) for the SMls were not entirely made based onChapter 5A of the Operational Manual dated September 1998, which stated that the nominated SP shouldbe selected from three shortlist consultant firms. In many cases, SPs were recommended by the SMIsthemselves. These problems were further investigated by the Bank team where MC eventually providedsome documents, showing that while SMls recommended the SPs, there were actually three in the list forcomparison. The explanation was accepted by the Bank but the issue was raised in the Bank report (aidememoire) which noted that future contracts should be implemented in a stricter manner. Subsequentex-post review showed that this problem was corrected.

Most implementing agencies were also confused in the procurement process that had conflicts between thenational procurement law as stated in Keppres 16/1994 and the procurement arrangement as agreed in theLoan Agreement and the Bank's procurement guidelines. For example, the implementing agencies claimedthat following Keppres 16/1994, all procurement process should include a pre-bid meeting which was inconflict with the Bank's procurement guidelines of not allowing such meeting for goods procured underICB. In addition, many procurement packages were conducted under national shopping and NCB followingthe Indonesian national procurement law, without prior consultation with the Bank.

Lack of qualified and professional Service Providers (SPsM. Although the MC went through the process ofselecting SPs, many of the selected SPs were still unfamiliar with providing good engineering or technicaldocumentation reports, including the manuals at the level reauired by SMs. Corrections and revisionswere made in many reports to counter this problem, which became very time consumning and resulted insome implementation delays. Disbursement process and procedures were one of the critical issues inimplementing the DAPATI grants for the SPs, many of whom were not familiar with the invoicing processand procedures. On the average, it took almost four to five weeks to disburse the invoice for each of thecontracts.

DAPA TI Program: There was a prolonged delay in the implementation of this component. To begin with,the MC came on board only in Sentember 1997, with a delay of more than one year. Then it took a few

- 14 -

months for the MC to preDare the first five contracts for the Bank annroval. When these were submitteddue to non-compliance of the DAPATI Implementation Manual by the MC in the preparation of thesecontracts, there were a series of comments/clarifications and exchanaes of communications between theBank and MolT/MC and consequently the approval by the Bank was delayed.

The financial problems that the country was facing thwarted the progress of this component Demand fromindustry for this nrogram was low for two reasons: (a) significant denreciation of the Runiah made itdifficult for SMIs to come up with the requisite matching funds of 25-50 percent (depending ongeogranhical location) to finance consultant services; (h) the shortage of financiLng for SMT industrieshalted subsequent investments identified by the consultants. Due to the above problems, the Program'stargeted contract numbers were reduced from the nrioinal 600 to 130. The amnount allo-ated to thiscomponent as well as the contract amount to the MC were reduced correspondingly.

5.4 Costs and financing:

SAR had estimated the project cost at US$60.90 million with a Loan amount of US$47.0 million.However, in light of the 1997 economic crisis, the Loan was subjected to restructuring together with allother IP.RD projects in response to GOIs call for portfolio restructuring across the board. The impact onthe ITDP was to reassess the components and cancellation of some project activities which resulted in loancancellation/savings amounted to $8.5 million, bringing the total loan amount to US$38.46 million. At thetime of project closing, loan disbursement amounted to US$32.64 million, i.e. 84.7% of the reduced loanamount. The undisbursed balance of US$5.92 million will be cancelled.

6. Sustainability

6.1 Rationale #nr vstainhilitv ltinrtin-

Before making judgmnent on whether the project is sustainable, it is important to understand the objectivesof the project beyond what is literally outlined in the previous paragraphs. It is also imperative to take noteof the difference between the long term versus short term objectives. The long tern objective of this projectwas to make the R&D sector financially independent so that they would have the freedom to pursue theirresearch objectives such as being responsive to SMW needs and conumercialization of their research outputs,both of which will yield long term sustainability of the project as well as contributing to the economicgrowth of the country. However, to achieve this long term objective, the R&D institutes have to overcomemany existing issues, such as budgetary constraint, and the traditional orientation of working for thegovernment to do research for its own sake. They also have to overcome the mentality of doing high profileresearch yet with little relevance to the industry needs. This situation would not be changed overnight. Theobjective of this project is to start the process of changing the mindset of the government officials as well asthe R&D institutions themselves in the short run and prepare them to take on challenge of doing researchfor their own long term sustainability through commercialization of their research output. Many activitiesof the project which aimed at achieving the short term objective may not be readily quantifiable, yet themindset change of R&D institutions is shown throug'h their cooDerative efforts with other R&D centers andattaching more importance to intellectual properties issue as reiated to their research output. The project'sgreatest contribution was to initiate the process of change within the R&D institutions and prepare themtowards achieving the long-term sustainability goal.

The project activities that appear to be highly sustainable include: (a) continuation of the Policy AdvisoryUJnit (PAIJ) as a think tank for GOI: (b) continuation of a DAPATI-like nromram bv BPPTP (although on a

- 15 -

smaller scale); (c) continuation of some of the twinning partnerships such as LIPI/CSIRO andIRDCI/NCL; and (d) the process of business orientation of the institutions leading to spin-off companiesor the conversion of viable R&D units into Perians.

On the other hand, the issue of long term sustainability for new structures and systems created under theproject needs to be addressed. It is very important that all aspects of this issue are examined at the highestlevel in GOI at policy level. For instance, the salary of the 62 extension agents who have been placed in theeight outreach centers, were covered by the GOI budget, and the promotional activities, and technicalassistance were financed by the project. A clear plan as to how these activities will be sustained, improvedand grown in the post-project phase as well as long-term plans need to be drawn on a priority basis toensure long term sustainabilitv. This issue has been raised, but has not received any commitment from theimplementing agency. This will form part of the items recommended for GOI consideration after projectclosing.

Maintenance of a modem IT infrastructure established in LIPI infrastructure (Comrnonent JII will reauirecommitted resources, both human as well as financial. Thus leased lines will cost Rupiah 350,000,000 peryear which will include LIPI online-system. Hardware and software upgrades will reauire a furtherUS$50,000 per year. Further, there are hidden costs for continuous staff upgrade as well as staff trainingon new tools. GOI will have to commit funds to sustain these systems until such time that the institutionsare substantially financially independent This will require revision of the current budgetary restrictionswhich hindered their total freedom of seeking external earnings. The International Workshop on Scienceand Technology, which is taking place in June 2002, will focus on budgetary restriction issues and isexpected to result in positive spnnort by GOT through revision of some relevant budgetarv regulations.

Several skills that were acniiirrd hv the Tndonesian instihtuions either hv twinning arrangementS or hv theacquisition of software and hardware should be made available to the benefit of other institutions which didnnt acqlire these skills hv menns nf "skills snread" nroram in the nnct-TTTP nhae. Thiiz r.miid be pdAneby making a skills inventory, institutional demand inventory and a supply strategy. BISm recently sourcedthe skills from BPPT and LTPI, rather thnn from overseas, with aappy hnntnu rnnp. The TndornesianUniversity system could also benefit through such spread.

Finally, a Forum of Seven Institutes has been set up for the first time to discuss diverse issues connected-ith _,o __u.re of t1,- mnoti4t,tnn- Th.e. . ise of rrunn nore aun.ononA a.orr. in t..s of tbe

utilization of the income generated from the external sources should become one of the priorities of theIFor i ... I. Fora --- c __. one v. the o:gam-m.v the w.-natior-t WoA o __ n ScienceanTechnology is one step forward towards this direction.

6.2 Transition arrangement to regiilar operations:

No specific transition arrangements are envisioned. Regular operations would continue with the governmentbudget in the short run and hopefully, the institutions benefited from the proiect could progress towardstheir financial independence in the long-mn; twinning arrangements do not need additional budget and thepartnership is expected to continue even after project closing.

7. Bank and Borrower Performance

Bank7 I r.. g:

Satisfactorv. The nroiect's identification was well oreanized and satisfactorv. The Bank took into account

-16 -

the project's consistency with the govermment's development priorities and the Bank's CAS. The Bank,which had a harmonious team with a good skill mix, including staff with various professional backgroundsand a top-rated intemational scientist, brought state-of-the-art expertise into project design.

During preparation and appraisal, the Bank took into account the adequacy of the project design and allmajor relevant aspects such as technical, financial, economic, institutional, including procurement andfinancial management, environrmental, sociological, and consistent with the Bank's safeguard policies. Inaddition, during the appraisal, the Bank assessed the project's risks and benefits. The Bank had aconsistently good working relationship with the Borrower during preparation and appraisal. Nevertheless,Bank's performance could be better shown if the designed project objectives differentiate more clearly thelong-term as compared to short-term objectives.

7.2 Stupervision:

Satisfactory. The Banks performance during the implementation of the project was satisfactory. Over theSX year-s of project im,plementation, hUere were eigh.t supervision Lissions. Iil addition, tihe Banik's projectteam kept close contact with the implementing agencies through frequent electronic mail communications aswell as telephone conversafuios wiuiuil wure iounu to oe very eniecuve m resolving urgent issues. TheBank's client relationship was very cordial and productive. The Bank took a finn and dynamic role in thesupervision of the project implementation. Aide remoires were reguiarly prepared and transmitted, whicnalerted the government to problems with project execution and suggested remedies in a timely manner, inconformity with the Bank procedures. The Forrn 590s and Project Status Reports (PSRs) realisticaiy ratedthe performance of the project both in terms of achievement of development objectives and projectimplementation. wnenever aelays im implementatuon occurred, the BanK was able to define concrete stepsand a timetable for putting the program on track.

Even after project closing, the Bank's project team continued to work with the client on the sustainability ofthe project. The continued effort led to fruitfui result in obtaining support from the Bank for launching ofan International Workshop on Science and Technology in Indonesia which is beyond the project scope, anactivity which will enhance the project sustainability. The workshop is aimed to provide a venue for toplevel government officials as well as policy makers and research institutes to gain first hand experiencefrom the international scientists on the successful institutional transformation of similar R&D institutesaround the world It will also provide a forum for debate among government decision makers andinternational scientists in order to define the blue print for the future R&D development for Indonesia. Theworkshop has been scheduled to be held on June 17-18, 2002. This initiative is an added effort by theproject team.

In compliance with Bank's requirements, the project team included procurement and disbursement ex-postreview in every supervision mission, especially during the second half of project implementation, andwhenever problems were detected, the project team raised issues with the Project ManagementImplementation Unit (PMIU) and urged for remedial actions. For instance, in one case of rnisprocurement,the agency concerned was asked to use their own fund for the purchase and the agency complied byreturning the loan.

External consultants, with expertise in their own fields and vast experience on similar programs in otherregions, were used for specific aspects of project components. The Bank staff worked closely with thegovernment and the implementing agencies and provided them with extensive assistance, in terrns ofreviewing their technical documents and provided professional advice to the regulations affecting thebudgetary issues haunting the R&D centers. This effort has resulted in a fruitful endeavor of launching of

- 17 -

an International Workshop on Science and Technology, the main focus of which is to review internationalexperience of financing R&D institutions and how this could apply in an Indonesian context.

7.3 Overall Bank perforniance:

Satisfactory. Overall, the Bank performance was satisfactory during project preparation, appraisal andimplementation.

Borroiver7.4 Preparation:

Satisfactory. The Borrower's performance in the preparation of the project was satisfactory. During theprparation stage, the Bnrrniovr edisnpluyd ar., admint,p ]pup] nfrn.nmitmert to the nkopjtivpe nf *e project

and all major aspects such as technical, financial, economic, institutional, environmental and sociologicalfactor-s, icludin.g st-keholder com en Th.e gov-ernent officials and staff of the implementingagencies both worked closely with the Bank's project team on a continual basis. Although there were

as+L US wCCdiues ocrreAd1, the Barb tem raised +-he issues in ti.. and Lh. IVUs

were receptive to suggestions and took remedial actions.

7.5 Government implementation performance:

Satisfactory. Despite changes in administration, the govermment consistently maintained its commnitmentthroughout the implementation. It had established good project imnplementation units and was able toovercome many issues encountered during project implementation. Owing to the 1997 East Asian Crisis,the govemment frequently faced counterpart funding problems.

7.6 Implemnenting Agency:

Satisfactory. Performance of BPPT, LIPI, and MoIT was satisfactory. Overall, PMIUs of BPPT, LIPIand MoIT implemented their tasks satisfactorily. During implementation period, there were instanceswhere procurement and disbursement problems occurred. However, the PMIUs were always receptive tosuggestions and took remedial actions such as undertaking intensive trainings on procurement whicheffectively addressed the procurement problems. Project reports such as Semi Annual Report andMid-Term Report were prepared and submitted to the Bank on schedule. This led to successful monitoringof the programs as well as providing assistance to the supervision missions of the Bank in assessing theprogress against the set milestones.

7.7 Overall Borrower perfiormance:

Satisfactory. The overail performance of the Borrower was satisfactory.

8. Luessorns L'earr.ted

* In order that research institutions move away from being wholly government-funded and developself-financing capacity, the government should provide strong incentives for external revenuegenerating activities and allow the institutions to retain and accumulate their earnings, and determinethe use of these earnings with greater flexibility and freedom.

* The Indonesian institutions are capable of providing service to each other, rather than seeking it fromabroad. The "Team Indonesia" spirit developed during the project arnong the institutions in MoIT, LIPI

and BPPT should be spread to universities with suitable incentive schemes. The exnerience from thisproject with institutions learning from each other proved to be very useful and ended with good results.This lesson could be useful for future nroiects in the proiect design stages to explore the nnssihilitv oftwinning among the institutions within the country, rather than always seeking help from abroad.

* The Borrower should consider providing additional capacity building for implementing agencies on theBank's procu ement proceduires. This is especially important for n,w agencies imrnplemr.enting theBank's project for the first time. During the project implementation period, borrowers experiencedprocuremnent problems dule to thteir lack of untderstanding the Bank's Proculrem.ent Guidelines. It isimportant therefore to provide implementing agencies the necessary training up front, rather than whennroh1irme nr fiptpt'tpti dirirur imnmpmptnt,,tnn Ru nmuri,-ina fhp trmuninc, -, -Fr~^ t ...Hi .1 -- ,un-problms -- --Pce dIn ..I....m.e. n--..a . By p-z,g h., -- -, Up .iont _vi! also avoidpossible delays in project implementation and the case on misprocurement.

* The support required for SMIs is not in technology and software alone. Capital financing to the SMIsta aol. +it-n #- fi,16, -h.i,n,,bt th-n, ib- --n,irA ,n+1, -rn.on+ an - bnn, nnna-n-.t 0toen.able therr. to . jA .l a thy hae acq_ -* .l proj_. (e. 5 . t-L.-WlJy, r,.aage,,e ard-

marketing skills) is also very important and yet seemed lacking to many of the SMIs which haveI-eneA4fi+A Un +1A. tAA AAiAA D Ar ; - A;A- - - > - - A- -- +A] --A 4JsAoAI QAA _ TO ;- -U\.vtu uer - vJ - _um1 .1,o,v I *v at,.oo *V ..p. axiu .wu1VIU&Y luI t.1Vin 1 aparticularly important factor for their business growth. In this respect, the project should have beenintegrated in a broader strategy of assistnce VW A

9 Partner ('nnmmentQ

(a) Borrower/implementing agency:

It is pleased to say that the ITD project has been completely accomplished. When the GOI and theWorld Bank drew the objectives, we found that it is important for national industries especially Srvilsto improve their competitiveness through improved productivity, product quality, reliability anddelivery. Tne objectives of -i DP are stiil relevant for 5 - 10 years ahead. The project has pioneeredand contributed to improving Indonesia's R&D centers and facilitating the access of SMIs to thesetechnology services. The improved manufacturing and management skills of SMvis wiii, therefore, be astrong basis for competing in a fast changing environment. In our review, the project design isadequately efficient since it involves three Indonesian agencies (LIPI, BPPT and MUIT) prominent inindustrial technology development. The Bank's endorsement to bring them together in ITDP hasestablished better communication and productive cooperation, such as collaborative research,technology forun etc.

Despite the constraints that were faced during the implementation period and resulted in delays and, tosome extent, cancellation of several activities, the results were reasonable. From our perspective, theimplementing units persistently maintained their commitment to achieve the targeted outputs asspecified in the appraisal document. It is worth noting that the World Bank teamn consistently helpedsupporting the implementing units to keep the project activities on track.

Given the difficulties during the project implementation, it is amazing that there was no revision in theobjectives nor the components. There were several amendments to the Loan Agreement focusing mainlyon budget restructuring. Substantial adjustment seemed less necessary, though revisions of few projectoutputs were made. In fact, problems facing project implementation agencies varied widely from macroto micro factors. At macro level, the economic crisis which hit Indonesia in mid-1997 brought highdepreciation of Rupiah against dollar. As a result, local budget provided for counterpart financingbecame limited. The budget difficulties had forced project restructuring that inevitably affected theimplementation schedule. The political shift in 1998 continued to 2000 and had also contributed to the

-19-

:UriM1 iriSU1UVV111VIL VI tUV pr1ojecL dLaUVLUVb. rUIIUwUir iLVqUVUL sLL inI LuhV dUU1i1M, LUML WerI NrVfeLU11

personnel mutations. The new officers who replaced the predecessors required some time to getacqaaLir WIUI ' Uie pjeJUcA. Lis bUUvio-tisly keptL ULU aczUV1L1U Ub.ULhU ULU. UILCLLUU sciiUULe.

AtIL a LLucrO level, therIC werC inteUma aUU exteria radwUacs founu uuring the imnplemen'tation period.Internal factors were ascribed to the shortage of competent administrative as well as managerial staffsto carry out daily project management inicluding database maintenance, and institutional constraintsappeared to be fraught with bureaucratic practices. External factors were probably due to the WorldBank reorganization pian in i997 and the so-cauled 'strategic compact' that was reiated todecentralization of operation functions from headquarter to regional office. Surprisingly, the Bank'sregional office in Indonesia became a better site in i998. inese two issues seemed to restrain'just-in-time' decision related to Bank's approval to project documents. Although the issues seemedless cruciai, they stiii affectedc the impiementation schewule of some of the key activities especiallywithin that period.

In view of consistent efforts carried out by implementing units, we can say that the overall projectperformance is satisfactory. This obviously is based on the assessments of the tangible and intangiblebenefits gained from the project. One of the important benefits can be seen in the case of twinningarrangement. This was relatively a new experience for Indonesian R&D institLtes. Indeed, theexperience has effectively brought positive impact to them as they gained knowledge on better R&Dmanagement practices from their overseas partners. In the future, LIPI, BPPT and MOIT may exploretheir own best practices, which would enable them to provide better services to Indonesian industry inorder to compete at international market. Another new experience with a substantial benefit is DAPATIprogram. This program has resulted in a growing quality awareness in SMIs whereas in the past, lessattention were addressed. It has also contributed benefits to the SMIs through productivity and revenueincrease. In the future, SMIs may have better competence to win the domestic markets especially whenfree trade area becomes effective.

Satisfactory rating is not given merely by assessing the positive impacts, but also considers the lessonlearned from unexpected results. One example is related to procurement skill. It is apparent thatpersonnel involved in this project required a comnprehensive understanding on the Bank ProcurementGuidelines and the Procurement Plan. In the first half of project period, the procedures were consideredstrict and inflexible. This was due to insufficient understanding on the procedures, and consequently,many unnecessary steps were taken. We were pleased that the Bank actively conducted severalprocurement training courses, throughout the implementation period and opened online consultation onprocurement process. As a result, some activities could effectively be accelerated at the second half ofimplementation period and resulted in better outputs and achievements.

Considering the performance achieved, some of the project activities are perceivably sustainable. Werealize that it is important to thoroughly address the issues of socialization, institutionalization andimplementation linked with various aspects such as economic incentives, institutional regimes,education, and labor. Sustainability also requires support on financial commitnent. However, thesustainability of the project would be much higher if the Bank provides even further assistance.

In our opinion, DAPATI is a prospective program for future arrangement with the Bank. ThroughDAPATI, it has encouraged a cooperative relationship between public and private sectors, in particularwith SMIs. Awareness on the importance of technology services requires to be developed andsupported with seed capital for technology investment. This will facilitate SMIs to expand theircapacity and competitiveness.

- 20 -

In addition we observe that ex-post review through drawing up Implementation Completion Report isnot entirely sufficient, since it has a limited perspective to extensively assess the long-term Impacts.Therefore, we suggest that in the future the Bank could facilitate a follow-up evaluation for anindividual project covering issues in economic as well as non-economic impacts.

Last but not the least, we would like to express our deep gratitude to Bank's task managers of ITDPfor their assistance, contribution and good cooperation which led to the success of this project. Weacknowledge their enthusiasm and optimism to overcome the impediments related to 'Swadana' issues,which had hindered the (flexibility of the R&D institutions and thereby the) effective provision oftechnology services by them.

(b) Cofinanciers:

nihc -nrt--t. /Alfln,a cor)

10. Additional Information

- 21 -

Annex 1. Key Performance Indicators/Log Frame Matrix

Outcome / Impact Indicators:

I lndieatorlMatrix Prolated In last PSR A euallLatest Estimate

|Not applicable

Output Indicators:

Indicator/Matrik Projected In lastPSR ActuaULatest Estimate

I. INSTITUTE OF NATIONALMETROLOGY (KIM-LIPI) 98t99 99/00 0o 01 98/99 99/00 00 '01

1. No. of measurement fields of KIM-LIPIistad in BIPM Globha MRA 2

2. No. of qualified assessors (cumulative) | 30 27 29 29 29

3. No. of calibration labs accreditedtD 25 30 30 17 20 25 40ISO/IEC Guide 25 or ISO 17025 (cumubtive)

4. % of calibration labs showing acceptableinter-comparison results 80% 90% 90% 50% 59% 65% 80%

5. Longest delivery time of calibration at

KIM-LIPI (weeks) 4 3 3 5 5 4 3

II. R & D CENTER FOR APPLIEDCHEMISTRY (RDCAC-LIPI) 97/98 98/99 99/00 '00 01 97/98 98/99 99/00 '00 '01

1. Self-financina Ratio 43.7% 49.4% 30% 35% 26.5% 26.6% 22% 16% 43%

2. Numberofnewclientscontacted 15 18 30 35 16 34 28 285 376

3. Number of conacts signed: 6 7 8 1Research contracts 4 6 8 10 7 11 9 11 9

CTruaining 12 12 12 12 2 3 4 1 2Training: ~~~ ~~~75 75 25 25 6 1 1 9

Mateuial analysis: 3 12 26

4. No. of contracts successfully completed: 5 7 8 10 6 7 9Research contracts: I 8 8 10 5 4Consultancy: 12 12 12 12 6 11 9Training: 75 75 25 25 25 32 34 26Ma'.eda anagys:si

2300 2900 1500 2000 890 1217 1156 844 16165. Rev.enue ge~neratAd firm teGh sAven.aes f

0-.43 0.49 0.30 0.35 0.26 0.26 0.21 0.17 0.286. Overall income to expenditure rato

7. Number of technologies licensed I 2 2

150 300 15 50 15 15I8. Revenue generated from technobly

licensing480 510 200 100 80 15 15

19. Value of production from commercialized |1 2 2 2

10. Number of laboratories accredited |

IMNuMt='4'I3MV~ I 0I T I | CI II

STRENGTHENING (MSS-LIPI)

1. Self-financing Ratio 96/97 97/98 98/93 99/00 '00 01 196197 97/98 98/99 99/00 00 '01

I2. Number of companies directl 11i% 8% 11% 13% 15% 18% 11% 8.2% 8.2% 10% 10% 11%contacted

13. Number Of DroDosas submitted 25 25 75 250 300 400 1 - 14 75 200 300 500

4. Number of contracts signed 10 5 10 20 29 40 - 2 12 5 21 11

5. Number of companies informed 5 10 2 8 9 16 - 2 1 5 6

1250 400 400 1000 1125 1500 62 258 700 8S0 15

IV. SEMARANG INSTITUTE -MOIT97/98 98/99 99/00 00 01 97/98 98/99 99/00 00 '01

1. Self-financing Rato 58% 63% 57% 57% 57% 85% 71% 64% 92% 57%

2. No. of New Clients | 30 45 60 70 80 52 46 38 60 120

3. No. of Contract actually signed (design 70 85 100 110 120 145 355 293 350 410engineering, testing, technologydevelopment, consultancy)

4. No.ofContractsuccessfullycompleted 70 85 100 110 110 145 264 53 40 390

5. Revenue generated (million nupiahs) 1156 1491 2003 2074 2100 1949 1393 1299 1554 1687

6. Total Expenditure (million rupiahs)* 1981 2431 283 3001 3359 3032 2450 i912 1912 2316

17. Income to Expenditure Ratio | 0.58 0.63 0.57 0.69 0.63 0.66 0.57 0.68 0.68 0.72

8. No. of Technology licensed 1 1 1 1

9. Revenue generated from Tecnnoiogy 10 10 10 10licensing( million nupiahs)

1 u. N4o. of patents fliied in nationaiy 1 1 1 1I I 2

11. No. of laboratories accredited nationally 1 1 1 1 progress progress

STANDARDS (PUSTAN) - MOlT 98/99 99/00 '00 '01 98/99 99/00 '00 '01

1. No. of co. in quality awareness program 3000 3000 3163 3163 completed (C) C

2. No. of co. implementing quality programs 600 600 6 _ _ C

3 No nfrn seekinno IOnn mrtifi.in I 4 40 40 40 C Cthrough ITDP supported KAN Program

(b). Trainings of Extension Agents &Establishing Outreach Centers Done Do

1. Market Study completed/outreachprogram planned Done Done

2. MSTQ Extension Agents trained 800 800 865 891 C C

3. No. of SMIs assisted by agents

(c). Upgrading Laboratory Management & Done All 5 Done All 5 C CIntercomparison Testing

1. Preparation of strategic plans for 5 labs. All 5 All 5 All 5 C C

-23 -

2. No. of abs wtthin acceptable limits Of 3 All 5 All 5 C Cinter-comparison

3. No. of labs In conformlty with ISO/IEC 5 C C CGuide 25/DSN01

4. No. of bbs Intemationally recogntzed 5 4 C Cand/or accredited

5. No. of tabs nationally recognized andhoraccredited

The numbers are cumulatve

vI iNST!n ITF FOR RP.& OF CHFMIGALINDUSTRY (IRDCI) - MOIT 97/98 98/99 99/0 '00 01 97i98 98/99 99/00 '00 '01

I F. Fnancig Ratio 36.6 % 34.9% 49.1% 36.3% 24.7% 57.7% 45.5% 49.1% 36.3% 24.7%

12. Number of new dlent contacted for | 80 80 90 80 70 70 54 94 67 90business linkage .

3. Numberofcontradssrted 45 40 50 55 so 37 10 79 49 72

4. Number of contracts successfully 45 40 40 50 45 22 9 67 43 70completed

829 1500 1766 1143.8 1019.8 2107 1360 2033 1267 1164.65. Revenue generated (Rp. m)

2266 3406 4090 3326 4582 3650 3270 4139 3494 47066. Total income (Rp. m)

0.37 0.41 0.49 0.36 0.24 0.58 0.41 0.49 0.36 0.247. Income to expenditure ratio (%)

8. Number of Technology lcensed 2 2 1 4 1340 500 450 85 347 536 275

9. Revenue generated from technologylicensing (Rp. m)

1 0. Number of laboratories accreaitea 1 1 1 5 5 5-natonally 1 1 1 4 4-intemationally

AGENCY FOR ASSESSMENT ANDAPPLICATION OF TECHNOLOGY (BPPT)

VIl. TECHNOLOGY MANAGEMENT (TBIS) 99 99s 200001| 98/99 99/00 2000/01

1 Self-financing ratio 5% 7.5% lU'%i1 5% 4% i3%2. Business Development 0 200 250 0 190 200

1~~- AA rlf~.0 55 70 0 74 704. IP Management (patent)5. Marketng Communicatons 1,000 2000 3000 0 2,000 2500

6. Operator Autonomy 50% 50% .75% 50% 50% 50%2000 2001 2000 2001

17 DIP (Dev.Budget) 3'$,32R 1 35 0 63354 528,000 6983569833,5148. DIK (Routtne Budget) 63,289,928 86853014 63,289,928 86,853,0149. DIKS+DURK (Contract Research 11,791,037 10963877 11,791,037 15,530,755

DAPATI PROGRAM 97/98 98/99 99/00 00/01 97/98 98/99 99/00 00/01(i) No. of Contracts sgned 0 10 70 130 0 4 56 148(iJ)No.r tdoo .......wd s | |4 8

End of projectNote: i. Some indicators were designed only during project implementation and wee thefore not in the original SAR. Also, projected numberscould be difrerent during project implementation since additional infbrmation was available to refine the original projection.

2. During CY 2000, OI changed FY from April-Mamh to January-December. As a result, information contained in 200(1 is only for 9 months from

April to December.

Annex 2. Project Costs and Financing

Project Cost by Component (in US$ million equivalent)Appraisal Actual/Latest Percentage ofEstimate Estimate Appraisal

Project Cost By Component US$ million US$ million

IMSTQ 4.70 3.76 80DAPATI 14.40 5.34 37R&D Mgmt 32.60 28.22 86.5Institutional Strengthening 5.00 4.61 92.2

Totai Baseiine Cost 56.70 41.93Physical Contingencies 2.20Price Contingencies | 2.00 jj

Total Project Costs | 60.90 I iTotal Flnancing Required 60.90 41.93 |

Project Costs by Procurement Arrangements (Appraisal Estimate) (US$ million equivalent)

Proruremeht MethodExpendIture Categry- ICB N_ , te 2 N.B.F. Total Cost

---- v~%tner1. Works 0.00 0.00 0.00 0.00 0.00

(0.00) (0.00) 1 (0.00) I (0.00) (0.00)12. Goods 8.84 | 0.00 1 7.74 1 0.16 | 16.74

I 1 (7.95) (0.00) (5.30) (0.00) 1 (13.25)13. Services I 0.00 0.00 44.20 0.00 | 44.20L Mlsenllnneoun (0.00) (0.00) (33.69) (0.00)' (33.69)4. Miscellaneous 0.00 0.00 0.00 0.00 0.00

L I I (0.00) (0.00) (0.00) (0.00) (0.00)

I' Mlsce'ganeous (0.00) ).00 1 0.00.00) _ _.00_ (0.00)6. Misceiianeous j 0.00 j .00 U.U0 j .00 I _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I (0.00) 0 ( 00) (3 . 99) I (° °0 (° °

Total j 8.84 | 0.00 51.94 J 0.16 60.940 | ~~~~~~~~~(7.95) (0.00) (38.99) (0.00) (46.94)

Project Cost by.. Pr .cur.m.n A0 rrangemen,sLC 1/A.--111 -OL CCI 4- Atc, /1 IC!4D;II;At r.rU..;IslAIL

f Expenditure Category I Procurement Method N. B . 1 Tota ! G-_ _ -_ -_I - _NCB Other 2 _ _ __ _

|1. Works | 0.00 I 0.00 I 0.00 I 0.00 I 0.00 I

II (° °°) I (°-°°) I (° °°) I (° °°) I (° °°) I12. GoodI 112.77 1 0.86 1 I 1 3! 1 I I Q1AI

3. Se s ((12.77 0.86) (1.65 (0.00) _ (15.28)

,J. QteFVI%;t% v. Y) I 21.57 (0.30 (22.6

(21.57) (0.29) (0.80) ((.00) (22.66)

4. Miscellaneous 0.00 0.00 0.56 (.00 0.56

(0.00) (0.00) (0.56) (0.00) (0.56)

5. Miscellaneous 0.00 0.00 0.00 0.00 0.00

(0.00) (0.00) (0.00) (0.00) (0.00)

6. Miscellaneous 0.00 0.00 0.00 0.00 0.00(0.00) (0.00) (0.00) (0.00) (0.00)

Total 34.34 1.15 3.01 3.43 41.93(31.d (1.!5 (I3.01) ( ° °°I ') !) (38.50)

" Figures in pa.renthesiq are the amronts to be finaneed by the Bank Loan. All costs inriudep rnthigencies.

21 Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staffof thie project management ofi-ce, training, technical assistance services, and incremenial operating costs reiated to (i)managing the project, and (fi) re-lending project funds to local goveniment units.

Proigect Financing by rnmnpnnnt /in I st p: millinn aeniuivalnt)n

I (Ci1,imflT1pnt I Annraisnl Fqtimate A'tuAVI/ILtpt Estimate Percentage of Appraisal

|____________________ |Bank Govt. CoF. Bank Covt. CoF. Bank Covt. | CoF.

MSTQ 4.40 0.80 0.00 3.07 0.69 69.8 86.3 0.0DIAPATI 8.40 0.00 6.20 5.04 0I00 ).3(! 60.0 0.0 4.8

R&D Management 29.80 5.80 0.00 26.26 1.96 88.1 33.8 0.0Institutional Steghnn 4.40 1.!0 0.00 4.09 0.52 93.(! 47.3 0.0|TOTAL | 47.00 7.70 6.20 38.50 j 3.17 0.30 81.9 | 41.2 | 4.8

Co.F is Industry contribution

Annex 3. Economic Costs and Benefits

Not Applicable

Annex 4. Bank Inputs

Stage of Project Cycle | No. of Persons and Specialty Performance Rating( * , F.rPnnnmist I FMR Ptc.) I I..n+ 1

|______________Month/Year |Count Specialty ... Progres Objective

identificationiPreparationOctober/1993 3 1 Policy & Institutional Issues

Specialist, 1. Metrology &

Standards Specialist, I R&DSnecialist

April/1994 3 1 Policy & Institutional IssuesSpecialist, I Metrologv &Standards Specialist, I R&DSpecialist

November/1994 4 1 Policy & Institutional IssuesSpecialist, I Metrology &Standards Specialist, I R&DSpecialist, I Consultant

February/1995 5 1 Policy & Institutional IssuesSpecialist, I Metrology &Standards Specialist, i R&DSpecialist, 2 Consultants

Appraisal/NegotiationJune/1995 5 1 Policy & Institutional

Issues Specialist, IMetrology & StandardsSpecialist, I R&D Specialist,I Consultants, I Operations

l l I ~~~~~~~~~~~OfficerSupervision Officer

June/96 4 1 Policy & Institutional S SIssues Specialist, IProcurement Specialist, IMetrology & StandardsSpecialist, I ProjectImplementation Specialist

I ~~~November~/96 1 . Iruoiicy C.TlosILUtnl Issues I S US

Specialist, I Metrology & 'Standards SnpecialiQt I Pr-ject

Pi-. niqrv/97 2 Implementation SpecialistI Februa ry/97 1 2 1 ! POlicY & Instittitiona! lIses S S

Specialist, I ProjectImplementation Specialist

November/97 4 1 Team Leader, I RSI Sector S SCoordinator, I R&D Specialist, IMSTQ Specialist

March/99 4 1 Financial Specialist, I RSI S SSector Coordinator, I R&DSpecialist, I MSTQ Specialist

October/99 | 5 1 Financial Specialist, 2 S S

- 28 -

| Procurement Specialists, I R&D |Specialist. I MSTQ Specialist

August/00 4 1 Financial Specialist, I S SProcurement Specialist, IDisbursement Analyst, I R&DSpecialist

June/01 4 1 Financial Specialist, I S SProcurement Specialist, iDisbursement Analyst, I R&DSpecialist

11CRMarch/02 2 1 Task Manager, R & D

Specialist

(I) Staff:

Stage of Project Cycle [ Actual/Latest Estimate

No. Staff weeks | US$ ('000)Identification/Preparation I 124.3 310.8Appraisal/Negotiation 82.8 207.2Supervision 96.2 240ICR 9.6 4.8Total 312.9 806.0

- 29 -

Annex 5. Ratings for Achievement of Objectives/Outputs of Components

(1=-figh- SU=Suhstantial M=Modest. N=Negligible; NA=Not Annlicable)

RatingFn Macro policies OH O SU OM ON *NAOii Sector Policies O H OSUOM O N *NAn Physical O H O SU *M O N ONALI Financial O H O SU O M O N * NAO Institztional Development Q H 0 SU * M 0 N 0 NAEl Environmental O H O SU O M O N * NA

SocialLI Poverty Reduction O H O SU O M O N * NAI Gendler O H OSUOM O N * NA

LI Other (Please specify) O H OSUOM O N * NALi Private sector development 0 H O SUO M 0 N 0 NAEl Public sector nianagement 0 H 0 SU * M 0 N 0 NAEl Other (Please specify) O H OSUOM O N * NA

30 -

Annex 6. Ratings of Rnnk and Bonrrnwor Perfnrmance

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory)

6.1 Bank performance Rating

I Lending OHS OS OU OHUF-i .. r....- Lie' -h C II r LiIIIL...J \0

9CIVOIJlV A 40 l I '. LI '3 i IL

O Overall OHS OS O U O HU

6.2 Borrower performance Rating

LI Preparation OHS OS O U O HUO Government implementation performance 0 HS O S 0 U 0 HUOlI Implementation agency performance O HS O S 0 U 0 HU

FO Overall OHS OS OU O HU

- 31 -

IAnnex 7. LJi'st of S-upporting IJocut-ien'Ls

1. Aide Memoirs, Back-to-Office Reports, and Project Status Reports;

2. Project Progress Reports;

3. Consultant Study Reports financed under the Project;

4. Borrower's Evaluation Report dated March 15, 2002;

5. Consolidated Report on the Evaluation and Monitoring of ITDP, dated February 2002, prepared by theNational Developing Planning Agency;

6. Mid-Term Report on the Evaluation and Monitoring of ITDP, dated August 15, 2001, prepared by theAgency for Industrial and Trade Research and Development; Ministry of Industry and Trade;

7. Indonesia: Industrial Technology Development for a Competitive Edge, dated December 1996 (ReportNo. 18816), by Darius Mans, Sr. Industrial Economist, EA3IP, World Bank.

8. Staff Appraisal Report for Indonesia: Industrial rechnology Development Project dated November20, 1995 (Report No. 14900-IND)

- 32 -

! M@,YANM4 @IcA4I 0 A INDONESIAr' \~~~~~~~~~~VI'EPACIFIC' .- MAIN ROADS

Andiaman ~ 'II~PNS0 SELECTE.D CITIES AND TOWNSSea] ulf ( (uffof 0 ' OCRAN | PROVINCE HEADQUARTERS

ThiIland Slu * NATIONAL CAPITAL

THAILAND SUILI-~ PROVINCE BOUNDARIESC tk u Seo~-~ -, S PRO- INTERNATIONAIL BOUNDARIES

7 ~~~~~~~_\, 0 -~~~~~~~~~~~~C I D I ACEH I IJAWA RARAT 22 KALIMtANTAN TttMUR2 SUtmATERA UTkRA 12 JAWA TE GMN 22 SU-AES1 U'TARA2 ZAD 1 Dl. YOGNAKARTA 24 OcRONTALOBcinda, /On -10 RIAU ARCHIPEIAGO' 14 JAWA DIODER 25 SULAWESI TENGAH

BtindslA ceh}, A nr.N,,\ S A,RUNEl t 4 StIMAIMRA RARAT T 5 KAU 26 SULAWESI SELATANA ERUNI~ELn Mal JA51I NUSA TENGGARA BARAT 27 SULAWES TENGGARA.1 ~~~~~,S _ ll'~ 6 ERNGKDUW 1"TA 7 NUSAJDNGGARATDAUR 2B MALUKU UJAPA\-z *Kdon 7 9 / w Natuntl<)\ /4 st f _ > Celet1es o Tatud SIUMAuERA SEAMAN AT TEPA 29 MALUKU

.' ~~~~~~~~~Cel ebes 0TIud LA DNG 19 KALLYLANTAN BARAT 32 PAPUA IIRIAN JAYA)M.&nf I Ntm. \ I S. S BNKA-BEUIJ',4G 20 KALUNA ITAN TENGAH

snOeuluet a Anont't9 ~ato*au Istnds 7 > 2<t t fnan Sec i3 10 )ANITEN 21 KALIMANTAN SELATAN

L Ss kon5R9 u1 E N efn g eaD2 d 4, Bd Sec Sin,etIt$2n Riau Is-ndN 23 *t-d ° tfl

gf _ X X~ W%b Itlp I'-n- O1Ž Arc :15 a

|N <~> G NPOIE 221 < MonNd O5,

Nias'\) ,so \ t Fa A L I- XorsAtdrai TeDI da t e Carp wrn U4 :Lqn> 7he Poon,ottionok SonP,orIndo I */ 28 '-6"S.8"6 'a i .,- ' Bo.mn0 ntor htip 20 &a U R I/~~II~dOfl0 ~ ~ gkaIpiRo~g ~\ ~kPalangkoraya 0 L S ie an

26 n~~ * Ceroat~~~ y~fa*falcomJoyapur

-~Benguu . \~~~ dari 0 R, P I AIN . AYA'1 <

Java Se aD Ujung Pcindang Muno I Ka, .,

Engg-', La B~~~~~~5 M, , oino 0 I

~ o'. honorArafuSeSe

(- \~~~~:N~~~ONEsI/t.. INDIA N 0 CEA ~~~~~~~~~~~~N D 0E

< IS' -~~~~~~~~~~~~~~~~~~~9- Carnentaria~~~~~~~~~~~~~~~~~~~~~A> ~ mLi,aTh,,~on an proanei b theFla Oeign oilof he W,IdBan

I N D I A N 0 C E A N~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~020'

IMAGING

Te:o.f No.: 23742I Type: RI