improving development results through excellence in … · improving development results through...

48032 v5

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

IMPROVING DEVELOPMENT RESULTS THROUGH EXCELLENCE IN EVALUATION

Review of IDA Internal Controls An Evaluation of Management’s Assessment

and the IAD Review

Volume V

Report on the Completion of Part IA Process Mapping and Effectiveness of Control Design

This paper is available upon request from IEG-World Bank.

2009 The World Bank

Washington, D.C.

©2009 The Independent Evaluation Group, The World Bank Group 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org E-mail: [email protected] All rights reserved 1 2 3 4 5 10 09 08 07

This volume, except for the elements contributed by group and institutions outside the Independent Evaluation Group, is a product of the staff of the Independent Evaluation Group of the World Bank Group. The findings, interpretations, and conclusions expressed in this volume do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. This volume does not support any general inferences beyond the scope of this evaluation, including any references about the World Bank Group’s past, current, or prospective overall performance.

The World Bank Group does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank Group concerning the legal status of any territory or the endorsement or acceptance of such boundaries.

Rights and Permissions

The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The Independent Evaluation Group encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly.

For permission to photocopy or reprint any part of this work, please send a request to the Independent Evaluation Group. ISBN: 978-60244-113-2 Independent Evaluation Group Knowledge Programs and Evaluation Capacity Development (IEGKE) E-mail: [email protected] Telephone: 202-458-4497 Facsimile: 202-522-3125 Printed on recycled paper

Acronyms and Abbreviations

AAA Analytical and Advisory Activities AC Audit Committee ACS Administrative and Client Support

Network AICPA American Institute of Certified

Public Accountants AROE Annual Review of Operations

Evaluation ARDE Annual Review of Development

Effectiveness ARPP Annual Review of Portfolio

Performance AS2 Audit Standard No. 2 BP Bank Procedure BPM Business Process Module CAS Country Assistance Strategy CDS Control Detail Sheet CODE Committee on Development

Effectiveness COSO Committee of Sponsoring

Organizations, of the Treadway Commission

CPIA Country Policy and Institutional Assessment

CTR Controller’s DEC Development Economics and Chief

Economist DPL Development Policy Loan ESW Economic and Sector Work GPN General Procurement Notice FM Financial Management FMA Fiduciary Monitoring Agent IAD Internal Audit Department ICR Implementation Completion Report IDA International Development

Association IEG Independent Evaluation Group

(formerly OED) IL Investment Lending

IRMF Integrated Risk Management Framework

ISR(R) Implementation Status (and Results) Report

IT Information Technology LOA Loan Department NGO Non-Governmental Organization OD Operational Directive OED Operations Evaluations Department OM Operational Memorandum OP Operational Policy OPCS Operations Policy and Country

Services PAS Procurement Accredited Specialist PCAOB Public Company Accounting

Oversight Board PCPI Post Conflict Performance

Indicators PMT Project Management Team PO Process Overview PPF Project Preparation Facility PS Procurement Specialist SME Subject Matter Experts QAG Quality Assurance Group QEA Quality at Entry Assessment QSA Quality of Supervision Assessment RMCVP Vice President, Resource

Mobilization and Co-financing RMFM Regional Manager, Financial

Management ROC Regional Operations Committee ROW Risk Opportunity Workshop SIL Specific Investment Loan SOX Sarbanes-Oxley Legislation TSS Transition Support Strategy TTL Task Team Leader RS Risk Scan VAA VPU Access Administrator VPU Vice Presidential Unit WBI World Bank Institute

i

Contents KEY TECHNICAL TERMS .........................................................................................III

PREFACE................................................................................................................... V

EVALUATION SUMMARY ....................................................................................... VII

1. BACKGROUND AND DESCRIPTION OF APPROACHES..........................1

Origins of the Study.................................................................................................1 The COSO Perspective ............................................................................................2 Integrating the COSO Framework into Bank Operations .....................................4 IEG’s Approach to its Evaluation ...........................................................................4 Summary of Approaches: Management Assessment and the IAD Review........7

2. MANAGEMENT’S ASSESSMENT .............................................................11

Background and Objective....................................................................................11 Management’s Method: From the IDA Charter to Policies to Business Processes ...............................................................................................................11 IEG’s Evaluation of Management’s Approach and Method................................17 Management’s Main Findings and Conclusions: IEG Comment and Evaluation.................................................................................................................................22

Management’s Broad Conclusions................................................................................. 22 Management’s Highlighted Deficiencies (paragraph 26 of its report)............................. 23 Management’s List of Additional Issues ......................................................................... 25

Findings from IEG’s own Analysis .......................................................................26 Issues Related to Controls ............................................................................................. 26 Issues Related to Management’s Descriptive Materials and Mapping........................... 27

3. THE IAD REVIEW AND REPORT ..............................................................31

Context for IEG’s Review of IAD’s Work..............................................................31 IAD’s Objective.......................................................................................................31 IAD’s Scope and Approach for Part IA.................................................................31 IAD’s General Observation and Key Issues ........................................................32

4. CONCLUSIONS AND RECOMMENDATIONS...........................................37

Overall IEG Evaluation...........................................................................................37

Evaluation Managers Vinod Thomas

Director-General, Evaluation

Ajay Chhibber Director, Independent Evaluation Group-World Bank

Nils Fostvedt Task Manager

Ian Hume Team Leader

ii

Boxes Box 4. Stages in the Study of IDA Internal Controls.................................................... 2 Box 5. Key Components in the Management Assessment, IAD Review, and IEG Evaluation (Part I) ....................................................................................................... 7 Box 6. Overall Timeline for Completion of IDA 14 Assessment .................................. 8 Box 7. Building Blocks in Management’s Approach ...................................................13 Box 8. Management’s List of 30 Business Process Modules .....................................14 Box 9. Business Process Modules Excluded from Compliance Assessment.............15 Box 10. Summary of Principal Issues Identified by Management, IAD, and IEG ..........39

Figures Figure 1. The COSO Framework: Components, Objectives, and Risk Factors........... 3 Figure 2. Overview Scope Map of Management’s Assessment .................................16

Annexes ANNEX A. THE COSO FRAMEWORK .....................................................................43 ANNEX B. STANDARDS AGREED BY MANAGEMENT, IAD AND IEG TO BE USED IN ASSESSING DEFICIENCIES, SIGNIFICANT DEFICIENCIES AND MATERIAL WEAKNESSES ......................................................................................47 ANNEX C. ILLUSTRATION OF POTENTIAL INTERNAL CONTROL DESIGN WEAKNESSES..........................................................................................................50 ANNEX D. A TYPICAL BPM: DESCRIPTIVE MATERIAL........................................57 ANNEX E. DOES THE CLUSTER OF BPMS REPRESENT THE UNIVERSE OF IDA CONTROLS?.............................................................................................................65 ANNEX F. METHOD AND RESULTS IN APPLYING THE BUSINESS PROCESS TEMPLATE................................................................................................................70 ANNEX G. STATISTICAL APPENDIX ......................................................................73

Source Reports ATTACHMENT 1: MANAGEMENT REPORT ON ITS REVIEW OF IDA CONTROLS...................................................................................................................................83 ATTACHMENT 2: IAD REVIEW OF MANAGEMENT’S ASSESSMENT................117

KEY TECHNICAL TERMS

iii

Key Technical Terms Internal Controls: Controls, individually or in collective fashion, are structured means within an organization to enable it to achieve its business objectives, while addressing risk. Control instruments in-clude the control framework (in IDA’s case, the COSO framework), organizational checks and balances, published policies and required procedures, among others.

COSO Integrated Framework: A framework of management princi-ples (“COSO components”) in an organization which, when collec-tively operating as intended, will ensure the attainment of three key organization goals (“COSO objectives”), namely: reliable financial re-porting; operational effectiveness and efficiency; and compliance with laws and regulations, (or in IDA’s case with its charter and internal policies and procedures). The COSO components are: Control Envi-ronment; Risk Assessment; Control Activities; Monitoring and Learn-ing; Information and Communications.

Risk Focal Points: In the way the Management of the Bank and IDA have adapted the COSO framework to their own needs, four key points of risk which face the mission of the Bank Group—and are es-pecially relevant to IDA—have been defined and added to the COSO framework, namely: Strategy Effectiveness; Operational Efficiency; Financial Soundness; and Stakeholder Support.

Audit Standards: Criteria established by recognized accounting and audit bodies, which, in the course of reviews of internal controls sys-tems, enable the definition of deficiencies, significant deficiencies, and material weaknesses that may be revealed in those systems.

Business Process Modules (BPMs): Management chose to conduct this review of internal controls by identifying the main business proc-esses in which IDA is engaged on a daily basis in the course of its op-erations. These processes, 30 in all, covering IDA allocation, the CAS process, the main lending products (SILs and DPLs), and the fiduci-ary, contractual, safeguards and quality assurance processes that sup-port lending, were each mapped and described as separate business process modules, each containing the key internal controls that are the subject of the review.

Process Map: The flow chart that graphically depicts all steps in a business process module.

KEY TECHNICAL TERMS

iv

Key Control: A gateway and decision point, involving key units and IDA staff, in a given business process module, through which a busi-ness transaction being processed must pass. It is the effectiveness in design of these controls and the subsequent testing of the effective-ness of their operation that is at the center of this review.

Business Process Template: A standardized questionnaire and rating system used by IEG to provide quality ratings of Management’s method and approach in identifying, describing and mapping the business processes, and its method in assessing control design effec-tiveness and effectiveness of control operation.

Evaluation Panels: In applying its Business Process Template, IEG as-sembled 3-4 person panels, including specialists in the particular dis-cipline covered by the given BPM. The panels arrived at consensus judgments on what should be the ratings applied to each section of the module, according to their evaluation of the materials presented by Management.

Entity Level Controls: This refers to the control framework that gov-erns an organization at its aggregate level, emanating from central management down to the operating or business process level. In IDA’s case, the reference is to the elements of the COSO framework. Doing a controls review that started with an examination of entity-level controls, could be described as a “top down” approach.

Bottom up Approach: The approach adopted by Management in its assessment did not begin with a “top-down” entity-level review, but focused first on business processes at the operating level. Hence, it has been described as a “bottom-up” approach.

Walkthrough: An inter-active interview and review of process docu-mentation conducted by Management with relevant teams of IDA staff knowledgeable in a particular business process and its associated controls, with a view to verifying that controls are designed in the way described, and operate in the way intended.

Deficiencies, Significant Deficiencies, Material Weaknesses: Design flaws, omissions, or non-compliant operation of controls, discovered in the course of a controls review, denoting ascending order of seri-ousness. The precise criteria by which the three categories of material-ity are distinguished are explained in Annex B.

v

Preface In the IDA 14 Replenishment Report Bank management committed to carry out an independent comprehensive assessment of IDA’s control framework including internal controls over IDA operations and com-pliance with its charter and policies. Each part of this review is to be done in a three-phase approach: the first phase would be a self as-sessment by Management, to be followed by an Internal Audit De-partment (IAD) review and report on management’s self assessment, and an IEG independent evaluation of management and IAD work. This report contains IEG’s evaluation of the work done by manage-ment, and reviewed by IAD, for Part IA of the overall review.

The basis for the work done by IEG in its evaluation included: the re-port prepared by Management reflecting its assessment (Attachment 1); access to all the underlying materials that Management generated in its process based descriptions, definitions of controls and its “walk-through” testing of control design effectiveness; and the report pre-sented by IAD (Attachment 2).

Under the task management of Nils Fostvedt, this report was pre-pared by Ian Hume, with the assistance of a core consultant team, in-cluding: Dexter Peach (Strategic Advisor, formerly Assistant Comp-troller General for Planning and Reporting, GAO), James Campbell and Rosemary Jellish (Consultants, both former Assistant Directors, GAO) and Barbara Yale. The core team was assisted, in selected top-ics, by: Jed Shilling, Tribhuwan Narain, David Goldberg, and Mo-hammed Farhandi.

vii

Review of IDA Internal Controls:

An Evaluation of Management’s Assessment and the IAD Review

This report has its origins in a commitment that IDA Management made as part of the IDA 14 Replenishment process, in which it undertook “to carry out an independent, comprehensive assessment of IDA’s internal control framework, including internal controls over IDA operations and compliance with its charter and policies.” Management proposed, and the Board agreed, that Management would make an assessment of the controls, to be followed by an IAD review of the assessment and an IEG independent evaluation of both Management and IAD reports.

Contacts Director-General, Evaluation Vinod Thomas, 202-473-6300 Director, IEGWB Ajay Chhibber, 202-458-4219 Evaluation Manager Nils Fostvedt, 202-458-0719 Evaluation Author Ian Hume

Press contact Melanie Zipperer, 202-458-2902

Web site www.worldbank.org/ieg

Management decided that it would conduct its assessment within the COSO integrated controls framework, but it would divide its study into two parts: Part I would deal with compliance issues, and be focused on controls at the level of 30 business proc-esses, identified as representing IDA alloca-tion, CAS and IDA lending products, sup-porting contractual, fiduciary and safeguard processes, and quality assurance; Part II would deal with issues of operational effi-ciency and effectiveness, and would include an examination of entity-level controls, within the full COSO framework.

Management subsequently divided Part I of the assessment into two stages: Part IA, recently completed and the subject of the present re-port, covers Management’s ap-proach and method in identifying and map-ping the business processes that represent IDA operations, and assesses the effective-ness of the design of controls within these processes; Part IB, to be completed early in 2007, will deal with the testing of how these controls actually operate, compared to their design. Part II is intended for completion by Management at the end of calendar 2007, with the full IEG evaluation expected in early 2008.

This IEG report contains the evalua-tion made by IEG of the work completed by both Management and IAD in their re-spective assessments and review of Part IA. IEG conducted its evaluation using a com-bination of approaches:

• Verifying the legal, methodological, and operational basis for the approach taken in the Management assessment;

• Reviewing Management’s findings and conclusions;

• Participating as observers in a selec-tion of Management’s “walkthroughs” (verification interviews with knowl-edgeable Bank staff, concerning the actual design and working of key con-trols in the business processes);

• Creating an evaluation tool (a stan-dardized template), which generated a quality data base, enabling IEG’s analysis and evaluation of Manage-ment’s method in identifying and building the process maps and de-scriptive materials, and in conducting its assessment of the effectiveness of control design;

• Within the context of COSO, making an evaluation of the scope limitations inherent in Management’s approach, and their impact on the quality of conclusions that can be drawn at this stage of the review.

Findings Management decided the best way to

track the use of IDA resources, was to fo-cus its assessment at the transactions level on business processes. Doing so, it pro-vided a rigorous, transparent and concrete

viii

method for addressing internal controls, which was applied thoroughly and well documented. The assessment resulted in the production of 30 busi-ness process maps, accompanied by detailed de-scription for each module and its key controls. Overall, the assessment resulted in the amassing of over 700 pages of evidentiary documentation, and in identifying a significant number of potential deficiencies. This represents progress in develop-ing an understanding of IDA’s internal controls at the transactions level.

As evidence of this progress IEG would cite the following: • As a basis to test for compliance, Manage-

ment has made a credible linkage between the IDA Articles, the Bank’s policies and proce-dures, and the business processes identified to represent IDA operations;

• The mapped Business Process Modules have provided a concrete and transparent means of identifying, assessing and testing key controls;

• Management’s methods of mapping and as-sessing the BPMs were rated by IEG to be of a generally satisfactory quality, though with some notable qualifications relating to the treatment of risk, and the need to improve some of the descriptive materials;

• Management’s “walkthrough” method of veri-fying the accuracy of the selected business processes and testing the design effectiveness of their key controls was rigorous, compre-hensive, transparent, and documented to a sat-isfactory standard, with some qualifications. Management asserts that its approach gave a

representative picture of IDA transactions proc-esses, and that controls for the IDA allocation process as well as other controls over various as-pects of IDA lending are appropriately designed to suggest that IDA resources are allocated and used in accordance with the IDA articles, and in-ternal policies and procedures. It also documents that the approach succeeded in uncovering a sig-nificant number of specific controls-related issues. Of these, Management highlighted five it consid-ered to be most serious: • Difficulties with retention of and accessibility

to documentation needed to verify the opera-tion of key internal controls;

• Problems in keeping current the IDA OPs and BPs, which have not kept pace with the pace of change within the Bank Group;

• The policy framework for SILs being seen as too complex and cumbersome;

• Existing processes and documentary re-quirements for projects is seen by staff as on-erous and inefficient;

• A disparity in the frequency with which DPLs (always) and SILs (seldom) are sent for Cor-porate Review, instead of Decision Meeting processing. An evaluation of controls within the COSO

framework requires that all its components be ex-amined. Since this has not yet been done, is too early to make definitive conclusions on the state of the overall framework. However, from the de-ficiencies so far revealed, IEG considers that the issues highlighted by Management related to documentation retention, and the state of OPs/BPs are areas of potential material weakness. Management has initiated remedial programs in both these areas, and a firmer basis to draw con-clusions about their materiality will be in place once testing has been completed in Part IB. IEG has also had to take account of the trade-offs and implications of process-based, bottom-up method chosen by Management for its assessment, and the scope limitations this has implied. From this per-spective, notwithstanding the progress made, IEG found notable weaknesses in Management’s ap-proach: • Because conclusions on controls within

COSO cannot be made piecemeal, but only within the framework as a whole, staging and dividing the study has effectively postponed the ability to make definitive conclusions on the outcomes of each stage of the review un-til the overall (Part II) assessment has been completed.

• Even the staging of the study between Part IA and Part IB makes conclusions on control design (Part IA) difficult until Part IB has been completed, because final judgments on design effectiveness cannot be made until the operation of the control has also been tested.

• Separating compliance and efficiency and ef-fectiveness is really not possible in practice: many business processes and their associated controls are as much to do with compliance as with efficiency and effectiveness, and these are best treated together rather than in sequence. To illustrate, although management has fo-cused its efforts to date on assessing compli-

ix

ance, most of the potential issues it has identi-fied are related to efficiency and effectiveness.

• Other scope limitations flowing from the de-lineation of the study—in particular the deci-sion to deal with IT systems and field offices in Part II, have yet further limited the conclu-sions that can be drawn in Part I, especially given IDA’s increasing decentralization, and the growing importance of IT in maintaining the integrity of central controls.

• In taking 30 business process modules to rep-resent the totality of IDA operations, Man-agement has given a good representation of lending operations and the associated fiduci-ary processes; however, it has chosen to ex-clude AAA and other Knowledge Products, which IEG regards as a significant omission.

• In principle, it is possible that by completing the entity-level review during Part II, and ad-dressing the postponed parts of the frame-work, Management will be able to mitigate these deficiencies in approach by linking re-sults from the various parts together, to pro-vide an overall statement. However, this will depend on there being no changes in any ba-sic parameters: controls will be assessed at different points in time, and policies, proce-dures, systems, organization structures may change during this period. IEG therefore arrives at a mixed conclusion

on completion of this stage of the study: satisfac-tory progress has been made in defining, locating and assessing key internal controls at the transac-tions level, and the results have revealed a number of deficiencies and possible weaknesses in the un-derlying controls; on the other hand, the general approach and scope limitations applying to this stage of the assessment prevent any positive asser-tions being made now regarding the effective op-eration of the overall system of controls.

IAD was also positive in its findings of what Management had contributed to the Bank Group’s knowledge of its internal controls sys-tems, and the new information provided at the process level, stating that it provided a compelling baseline to streamline operations and improve ef-ficiency going forward. IAD identified eight key issues which it drew to Management’s attention: The exclusion of certain processes from the IDA processes selected; the fact that IT controls were not examined during Part I; the absence of fraud

and corruption controls in the scope for Part I; outdated OPs and BPs; the need to categorize and take remedies for deficiencies; the issue of docu-mentation retention and accessibility; the assess-ment of entity level controls; and issues relating to walkthroughs.

With the exception of the emphasis given to fraud and corruption and walkthroughs, all of these issues are also raised by IEG, with similar emphasis, and are covered in IEG’s overall evalua-tion. With regard to fraud and corruption, IEG believes that they (a) should be examined as part of the entity-level controls, and (b) were implicitly handled by Management in its process level ap-proach. IEG agrees with IAD that more explicit mention of fraud and corruption issues could have been made in Management’s process-level assess-ment. With regard to walkthroughs, Management and IAD have applied differing concepts of the term. In addition to these highlighted issues, IAD has also indicated that it has found a number of other deficiencies (55 in all). While these have been listed by broad type, IAD has not yet catego-rized these as to their materiality (i.e. seriousness of their possible impact on risk mitigation).

Recommendations Given the interim nature of the work so far

completed and the limited conclusions that can be drawn from it in relation to the overall system is-sues, IEG’s recommendations are focused on the issues to be dealt with in completing the remain-ing phases of the review, and on the broader con-trol framework issues that may emerge going for-ward. In this context, IEG makes six recommendations to Management (including one also to IAD), as follows: • Confirming the Validity of the BPM Cluster:

Management has argued, but has not conclu-sively demonstrated, that the core SIL proto-type module in the cluster of BPMs can be used as a proxy for all investment type lend-ing, because all ILs have the same controls as SILs. This proposition should be tested, and this could be done during Part IB. (para 2.18).

• Reform of the OPs/BPs: IEG considers this topic an area of potential material weakness, whose remedy Management should treat as a priority. IEG notes that Management has a stated strategy to address the problem, both

x

to streamline and to update the OPs/BPs. (para 2.37).

• Completing the Remaining Stages: IEG rec-ommends that preparation for the Part II stage should begin promptly upon comple-tion of Part I. It would seem useful to pre-cede this work with a work plan (which could be discussed with the Board), that could benefit from consultations between Manage-ment, IAD, and IEG, much as the Audit Standards were discussed under Part I. Part II should preferably be completed expeditiously, also because if it should be delayed, the con-trols parameters that were tested during Part I may have changed, and there may be diffi-culties in integrating the two parts of the as-sessment. (para 2.24).

• Resolving Specific Issues and Potential Defi-ciencies (Management and IAD): It is impor-tant that the several deficiencies uncovered by both Management’s assessment and IAD re-view, as listed and described in Annex C, be addressed during completion of Part IB. While some of these issues relate to lack of clarity in documentation, others to efficiency and effec-tiveness of controls, others are potential defi-ciencies in controls. It is the seriousness of the latter group—the materiality of their potential impact on risk mitigation—that must be ad-dressed before conclusions can be drawn on the state of the overall control framework. (paras 2.41,2.44 and 3.3, third bullet).

• Managing the Risk Framework and Extend-ing COSO: IEG believes the Integrated Risk Management Framework will need to be broadened to focus also on compliance and operations reporting, and in this context, the Bank may also consider adopting the recently extended version of COSO which provides for the addition of a new fourth objective (strategy—high level goals, aligning with sup-porting mission) and three new components to the existing five components of COSO: objective setting, event identification and risk response. (para 1.7 and Annex A paras 4-6)

• Mainstreaming Internal Controls Reviews: IDA should begin considering the value of adopting a policy requiring: (1) ongoing monitoring and reporting on internal controls in the course of operations for all three COSO objectives; and (2) separate evalua-tions and reporting as necessary. Attachments to the Executive Summary:

Given the relative complexity of this three-part review and the technical, detailed nature of the is-sues examined and the findings arrived at, IEG has provided tabular summaries of both the ap-proach and method of Management, IAD and IEG respectively (Box 1 and 2 below, extracted from Chapter 1), and of the main findings and po-sitions taken on the key issues (Box 3, extracted from Chapter 4, paragraph 4.6).

xi

Box 1. Key Components in the Management Assessment, IAD Review, and IEG Evaluation (Part I)

Management Assessment IAD Review IEG Evaluation Part IA

Define Approach and Method Business Process Based Fiduciary Focus Partial COSO Other Scope Limitations

Review Assumptions, Criteria, Methodology Criteria for Inclusion/ Exclusion Review process Test Methodology

Establish Framework/Tools The COSO Framework Business Process Template COSO Template Implications of Scope Limitations

Identify BPMs, Key Controls 4 Umbrella Areas 30 BPMs 114 Key Controls

Review Use of BPMs Criteria for Selection Definition of Key Controls Review of Process


Verify Mapping, Assessed Design of Key Controls Match Risks with Key Controls Conduct Walkthroughs Assessment of Design Effectiveness

Review of Management Assessment Attend Walkthroughs Review Assessment of Process and Design Effectiveness Apply Deficiency Tracker

Evaluate Individual BPMs Rank for Significance and Risk Provide Quality Ratings for Documentation and Mapping Assessment of Design Effectiveness Attend Selected Walkthroughs

Conclusions for Part IA Qualified Assurance

Conclusions for Part IA Opinion Postponed

Conclusions for Part IA Evaluate Quality of Management and IAD Conclusions Draw IEG Conclusions Implications for Part IB and Part II

Part IB Test Operation of Controls Conduct Audit of Controls Define Sampling Method Conduct Testing

Tabulate Findings Test Results Matrix

Review Testing of Controls Review test Methodology Review Process for Documenting Results Assess Process to Detect Fraud Review Deficiencies, Criteria

Evaluate Quality of Controls Tests Provide Quality Ratings for: Testing of Key Control Compliance Linkage to COSO Framework Conduct Independent Analysis of Management Exceptions data

Form Conclusions Statement of Assurance Make Recommendations

Unqualified Opinion or Modified Report

Overall Evaluation, Recommendations

Notes: BPMs—Business Process Modules

xii

Box 2. Overall Timeline for Completion of IDA 14 Assessment

Part IA Part IB Part II Management Report Aug 06* Dec 06 Sep 07 IAD Report Sept 06* Jan 07 Nov 07 IEG Report Oct 06 March 07 Jan 08 Source: Based on the Management paper to the Audit Committee and current esti-mates. * These reports were actually completed in early October 2006.

Box 3. Summary of Principal Issues Identified by Management, IAD, and IEG

Issues relating to approach and method

Management IAD IEG

A: Framework Issues 1. Bottom-up versus Top-down

Better start Top-down


2. Staging and Dividing the Assessment

Postpones Conclusions


3. Dealing only partially with COSO components

- Postpones Conclusions

4. Scope Limitations IT to be assessed in

Part IB

Optional; IT is part of Entity Level controls

B: Process Level Issues: 1. Definition of Objectives, Compliance

- Acceptable

2. From Articles to Key Policies and Procedures

- Acceptable

3. Linking OPs/BPs Explanations offered

- Only 50% linked to BPMs

4. Identifying BPMs - Acceptable 5. Quality of BPM mapping - Satisfactory,

some qualifications

5. The Cluster as Representing IDA Operations

Issue: Excluded Processes

a. Lending: Test ILs

b. Excluded AAA/KP

Issues relating to results: major controls issues

Highlighted Controls Issues By Management

1. Document Retention and Accessibility

Highlighted Deficiency


Potential Material

Weakness Continued

xiii

Box 3 (continued) 2. Current Status of OPs/BPs: a. OPs/BPs outdated, often not current b. Complex, disjointed policy framework c. Onerous, inefficient processes



Potential Material

Weakness

3. Disparity in Corporate Review SILs and DPLs.

Highlighted -- Highlighted

By IAD (3) 1. Outdated OPs/BPs

Highlighted Highlighted

2. Definition of Walkthrough Disputed Management Consistent with AS2 concepts

3. Fraud and Corruption Controls

Should be assessed at

process level

Start with Entity level

controls; could have been

more explicitly treated

By IEG (4) (i) No control over “subject to” disbursement changes; (ii) no assurance all refunds received; (iii) No mechanism to assure country safeguard documents redone if necessary; (iv) No Bank-wide log for procurement complaints

Highlighted

Issues relating to Results:

Documentation and potential control Deficiencies.

Highlighted Additional

Issues

55

Identified; Materiality

not yet established

Materiality should be

established during Part IB

1

1. Background and Description of Approaches

Origins of the Study 1.1 In the IDA14 Replenishment Report1 Bank Management “has committed to carry out an independent comprehensive assessment of its control framework including internal controls over IDA operations and compliance with its charter and policies” (paragraph 39 of that document). Annex B Table 3 of the document stipulated that this as-sessment should be undertaken by the Independent Evaluation Group (IEG, formerly OED). That document has been approved by the Execu-tive Directors.

1.2 This Review of IDA’s controls was discussed briefly at a Board meeting in May 2005.2 At that time, Management reiterated that—consistent with the practice that is being followed under the Bank’s COSO-based3 control framework—there should first be a self-assessment of the controls system, with a role for IAD, leading up to the IEG evaluation. IEG confirmed that it was prepared to take on the requested evaluation if the Board should so wish. As this was not in the IEG work program, there would need to be a non-fungible addition to the IEG budget for this purpose.

1.3 Management has since confirmed that the Review will be con-ducted in two parts (I and II). Part I will deal with internal controls over IDA’s compliance with its charter and internal policies and pro-cedures; and Part II with internal controls over IDA’s operational ef-fectiveness and efficiency. Each part will have three phases: first a Management assessment of internal controls; second, an IAD review of Management’s assessment; and third, IEG’s independent evalua-tion of both the assessment and the review. Following certain delays that Management encountered in completing the first part (on com-pliance), it decided further to divide this part into two stages (Part IA and Part IB), as described in Box 4.

1.4 The present report covers only that portion of the study to be completed under Part IA. It therefore deals with the assessment of in-ternal controls over compliance, to the stage of examining the identifi-cation and mapping of business processes, and the assessment of the

Evaluation Essentials This report focuses on

compliance controls and the design effectiveness of controls within business processes

Previous evaluations have found that internal controls need improvement to support changing processes and new initiatives

IEG evaluates the assessment by Bank management and the review by Internal Audit Department within COSO and uses agreed audit standards

IDA 14 committed to a review of internal controls

CHAPTER 1 BACKGROUND AND DESCRIPTION OF APPROACHES

2

design effectiveness of key controls within these processes. It will be a prelude to the assessment of the operating effectiveness of these con-trols, which is to be completed during the next stage (Part IB). Part II will then follow.

Box 4. Stages in the Study of IDA Internal Controls Management has divided its assessment into the following parts:

Part I—Compliance with IDA’s Articles and Policies -- This part has been split into (A) the identification of key business processes and controls and assessing the design effectiveness of the identified key controls; and (B) as-sessment of the operating effectiveness of the identified key controls through compliance testing.

A. This portion of the overall assessment identified and mapped the Busi-ness Process Modules (BPMs) and the key controls contained in each process. Then management reviewed the design effectiveness of the business processes and key controls involved to determine whether any significant deficiencies in the design of the key controls were iden-tified. Management recommended remedial actions to address the de-sign deficiencies.

B. Management intends to test a representative sample of prod-ucts/transactions to determine whether the key controls were ap-plied to the sample items as designed. Management could then de-termine whether there are significant deficiencies or material weaknesses in the operating effectiveness of the key controls and, if so, recommend measures for addressing any such deficiencies.

Part II—Efficiency and Effectiveness of Operations—Management plans to assess whether the existing internal control framework, including corporate governance and entity level controls, provides reasonable assurance that IDA’s operations are carried out efficiently and effectively, focusing on the processes and controls identified in Part I. In Part II management also plans to address the other scope limitations in Part I, such as information technol-ogy controls and the Bank’s operations in the decentralized field offices.

The COSO Perspective 1.5 The World Bank adopted the COSO internal control frame-work in 1995. COSO defines internal control as a process, effected by an entity’s board of directors, management, and other personnel, de-signed to provide reasonable assurance regarding the achievement of objectives in the following categories:

Reliability of financial reporting—relating to preparation of published financial statements;

Compliance with applicable laws and regulations—relating to compliance with applicable legal and regulatory framework,

This review covers only compliance controls and the design effectiveness of controls within business processes


3

which in the case of IDA is taken to mean its charter and poli-cies; and

Effectiveness and efficiency of operations—relating to effective and efficient use of resources in meeting business objectives.

Figure 1. The COSO Framework: Components, Objectives, and Risk Factors4

Information & Communication

Monitoring & Learning

Control Activities

Risk Assessment

Control Environment

Fina

ncial

Rep

ortin

g

Opera

tions

Compli

ance

Strategy EffectivenessO

perational EfficiencyFinancial Soundness

Stakeholder SupportCOSO Components

Bank’s Risk Focal Points

COSO Objectives



Control Activities

Risk Assessment

Control Environment

Fina

ncial

Rep

ortin

g

Opera

tions

Compli

ance

Strategy EffectivenessO

perational EfficiencyFinancial Soundness

Stakeholder SupportCOSO Components

Bank’s Risk Focal Points

COSO Objectives

1.6 To meet the three objectives, the COSO framework has five in-terrelated components that define the minimum level of quality accept-able for internal control and provide the basis against which internal control is to be evaluated. These internal control components, which apply to all aspects of an organization’s operations, include the control environment, risk assessment, control activities, monitoring and learn-ing, and information and communication. All five components must be present and effective in order for management to have reasonable as-surance that risks are managed to ensure the achievement of the or-ganization’s objectives. Management is responsible for developing the detailed policies, procedures, and practices to fit the organization’s op-erations and to ensure that they are built into and are an integral part of its operations, by conducting ongoing monitoring and, as needed, separate evaluations of internal controls.

1.7 A direct relationship exists between the three categories of ob-jectives—what the entity is striving to achieve—and components—the management dimensions the entity needs to achieve the objectives. These are depicted graphically in Figure 1, and are more fully de-scribed in Annex A. COSO is a dynamic framework which is being adapted continuously to changes in the global situation. Recent em-phasis in adapting COSO has been focused on better management of risk, and in 2004 COSO itself added a new strategic objective to the ex-

Internal controls focus on financial reporting, compliance with laws and regulations, and effectiveness and efficiency of operations

The COSO framework has recently emphasized risk management


4

isting three objectives (financial reporting, operations and compli-ance) and three new components to the existing five shown in the fig-ure above: objective setting; event identification; and risk response. This expanded framework has not yet been adopted by the Bank.

1.8 Given that the Bank has, for a number of years, assessed the internal controls over its financial reporting, and has had the external auditor attest to the quality of the assessment, the present review5 does not deal with financial reporting, but focuses on the remaining two COSO objectives, namely compliance and operations.

Integrating the COSO Framework into Bank Operations 1.9 The Bank’s Management has, since 1997, written internal, an-nual year-end reports on the status of the adaptation to COSO. IEG’s Annual Report on Operations Evaluation (AROE) 6 reports for 2000-2001 and 2002 addressed development effectiveness issues from the perspective of the COSO framework.7 The 2002 report noted the pro-found changes that had taken place in the Bank’s control environ-ment: new controls structures had been put in place and “a new cul-ture has taken root with respect to risk management,” but risk aversion appeared to have become a feature, and the report foresaw a need for what later became the Bank’s integrated risk management framework (IRMF). Under Control Activities, it saw the need for an accelerated conversion and updating of the Bank’s policies and pro-cedures, and under Monitoring it called for improved methods of evaluating Economic and Sector Work (ESW), grants and partner-ships. In the Information and Communication component it reported the major progress transforming the Bank as a Knowledge Bank, which had enabled rapid transfer of information on guidelines and best practice, and reported on the roles to be played by Development Economics Department and the World Bank Institute.

1.10 These themes are a relevant prelude to the present controls re-view. Being focused on development effectiveness, they preface the overall COSO-based approach that IEG is taking in the review, and the 2002 AROE stated the need for further developing the control en-vironment quite clearly: “… the drive to become a Knowledge Bank has engendered new initiatives and new processes, for which both the control environment and the evaluation framework have yet to be well defined.”8

IEG’s Approach to its Evaluation 1.11 IEG Objective: The objectives of the IEG evaluation for this Part IA Report must be viewed in light of the objectives of the overall review. IEG’s role in the overall review is to provide an independent

IEG previously has found a need for improved definition of the controls framework to keep pace with changing processes and new initiatives


5

evaluation to determine whether Management and IAD have pro-vided a reasonable basis for judging whether internal controls over IDA compliance and operations are in place and working; whether any material weaknesses and other deficiencies have been identified; and, as necessary, whether internal control corrective action plans are being implemented. Within that context, and for the purposes of pro-ducing a report on the status of work completed during Part IA of the review, IEG has aimed to:

evaluate implications of scope limitations and management’s phased approach;

evaluate management’s method for and completion of the mapping of business processes;

evaluate Management’s assessment of control design; and evaluate IAD’s approach to its work and its conclusions.

1.12 IEG Scope: IEG’s overall evaluation of both the management assessment and the IAD review offers an independent conclusion to the Board as to the degree of assurance with which the assessment and opinion presented respectively in the final reports by Management and IAD can be taken to be fairly stated, in terms of their giving reasonable assurance (or other conclusion) that IDA’s controls over compliance with its charter and relevant policies and procedures are effective.

1.13 IEG took the COSO framework and the audit standards con-sistent with that framework as the starting point for its evaluation. It assumed that the judgments regarding the effectiveness of the inter-nal IDA controls over compliance and operations had to be made against criteria contained in the COSO framework as a whole. At the same time, IEG recognized that management took an approach in Part I that has certain scope limitations. These limitations will have a bear-ing on the quality of assertions that can be made at this stage to the management and Board of IDA.

1.14 IEG notes that the key scope limitations in Part I are: the post-ponement to Part II of issues relating to entity-level controls; consid-eration of only two out of the five COSO components;10 the treatment of compliance only, and not efficiency and effectiveness of operations; and the postponement to Part II of the treatment of decentralized lo-cations and IT systems. As described in Chapter 2 and summarized in Chapter 4, IEG has evaluated the implications of these postponements in making its judgments on the overall quality of the management as-sessment and on the conclusions that can be made at this stage of the exercise.

The scope of this review is based on COSO as a whole


6

1.15 Audit Standards: Management had stated in its preparatory working papers that it intended to use audit standards similar to those used for its financial reporting (namely the general concepts of AS2 standard developed under the Sarbanes-Oxley legislation11), since these would provide due rigor and standardization. IEG be-lieved that this raised the question of whether it was indeed appro-priate to use the same standards for compliance and operations re-porting as for financial reporting, because the nature of the issues would be different. IEG therefore conducted significant research into this question. It was concluded that similar standards could be used, but that for compliance and operations reporting, assessing the mate-riality of deficiencies required more judgmental decision than for fi-nancial reporting. After consultations with both Management and IAD, agreement was reached both on the fact that all three parties would use the same standards, and on the precise definition for each. A description of the latter is given in Annex B.

1.16 IEG Evaluation Method: IEG has applied four principal meth-ods in making its evaluation:

It critically reviewed the available reports from management and IAD.

It conducted an independent analysis of the raw data gener-ated by Management’s assessment. This analysis addressed the quality and effectiveness of design of the underlying inter-nal controls.

IEG assembled evaluation panels for the purpose of rating each step in the assessment and review processes. The panels used an evaluation tool designed by IEG for this purpose, called a Business Process Template, which contained a series of standard questions on Management’s method of mapping and assessment of design of each business process module. This generated a data stream on the quality of Management’s method and approach to mapping and controls assessment.

IEG has reserved the option to conduct its own tests of the de-sign effectiveness of selected key controls, as a means of ob-taining verification independent from the results obtained by Management.12 This was found to be not necessary during Part IA. However, IEG did interview staff in selected units,13 in-cluding some that are involved in entity level controls, to gain a better understanding of the overall processes and controls that affect the business process modules included in Manage-ment’s assessment.

1.17 Advisory Panel: As is now normal for many of IEG’s major evaluations, a senior Advisory Panel will be invited to review and com-ment on the IEG evaluation report and will be requested to share its comments also with CODE and the Audit Committee (AC). The mem-

Management and IEG agreed on common audit standards

The IEG method used critical evaluation, independent analysis, and quality ratings developed by panels


7

bers of the panel for this evaluation are former Auditors-General, from India, Norway, and Australia, respectively. However, the services of the Panel will be invoked only when IEG has completed its draft evaluation report on Part IB.

Summary of Approaches: Management Assessment and the IAD Review 1.18 A description of the approach taken by Management to its as-sessment, and the key findings arrived at, is contained in Chapter 2, and a copy of the Management report is in Attachment 1. A description is given in Chapter 3 for the method, approach and findings of the IAD review, and a copy of the IAD report is appended at Attachment 2. Box 5 and Box 6 summarize the three approaches and the current timetable.

Box 5. Key Components in the Management Assessment, IAD Review, and IEG Evaluation (Part I)

Management Assessment IAD Review IEG Evaluation Part IA

Define Approach and Method Business Process Based Fiduciary Focus Partial COSO Other Scope Limitations

Review Assumptions, Criteria, Methodology Criteria for Inclusion/ Exclusion Review process Test Methodology

Establish Framework/Tools The COSO Framework Business Process Template COSO Template Implications of Scope Limitations

Identify BPMs, Key Controls 4 Umbrella Areas 30 BPMs 114 Key Controls



Verify Mapping, Assessed Design of Key Controls Match Risks with Key Controls Conduct Walkthroughs Assessment of Design Effectiveness

Review of Management Assessment Attend Walkthroughs Review Assessment of Process and Design Effectiveness Apply Deficiency Tracker

Evaluate Individual BPMs Rank for Significance and Risk Provide Quality Ratings for Documentation and Mapping Assessment of Design Effectiveness Attend Selected Walkthroughs

Conclusions for Part IA Qualified Assurance

Conclusions for Part IA Opinion Postponed

Conclusions for Part IA Evaluate Quality of Management and IAD Conclusions Draw IEG Conclusions Implications for Part IB and Part II

Continued


8

Box 5 (continued) Part IB

Test Operation of Controls Conduct Audit of Controls Define Sampling Method Conduct Testing

Tabulate Findings Test Results Matrix

Review Testing of Controls Review test Methodology Review Process for Documenting Results Assess Process to Detect Fraud Review Deficiencies, Criteria

Evaluate Quality of Controls Tests Provide Quality Ratings for: Testing of Key Control Compliance Linkage to COSO Framework Conduct Independent Analysis of Management Exceptions data

Form Conclusions Statement of Assurance Make Recommendations

Unqualified Opinion or Modified Report

Overall Evaluation, Recommendations

Notes: BPMs—Business Process Modules

Box 6. Overall Timeline for Completion of IDA 14 Assessment

Part IA Part IB Part II Management Report Aug 06* Dec 06 Sep 07 IAD Report Sept 06* Jan 07 Nov 07 IEG Report Oct 06 March 07 Jan 08 Source: Based on the Management paper to the Audit Committee and current esti-mates. * These reports were actually completed in early October 2006.

1. See “Report from the Executive Directors of the International Development Association to the Board of Governors, Additions to the IDA Resources: Fourteenth Replenishment, Working Together to Achieve the Millennium Development Goals” (approved by the Executive Directors of IDA on March 10, 2005).

2. The May 12 2005 discussion of IEG-WB’s FY06-08 work program and FY06 budget.

3. Committee of Sponsoring Organizations of the Treadway Commission, which published a report in 1992: Internal Control—Integrated Framework.

4. This representation of the COSO framework is what is currently in use in the Bank and IDA, showing also the Risk Focal Points, which have been added to the framework, following adoption by the Bank of the Integrated Risk Management Framework (IRMF) in 2002. For a fuller explanation, see Annex A.

5. IEG has verified that in recent years (2002-2005) the Bank published the management assessment report and the auditor’s attestation report in the An-nual Report, Volume 2, Financial Statements. This includes a transmittal letter, management’s discussion and analysis (covering just IBRD), IBRD financial

NOTES


9

statements and internal control reports, and IDA financial statements and in-ternal control reports.

6. Annual Report on Operations Evaluation (formerly OED, now IEG).

7. The AROEs for 2003 and 2004 also covered specific individual topics within the framework, namely monitoring and evaluation.

8. AROE 2002, page 31.

9. In management’s approach the focus was mainly on Risk Assessment (at the unit level) and Control Activities, and there was no direct focus on the other three components (Control Environment, Monitoring and Learning, and Information and Communications), which are postponed until Part II.

10. Auditing Standard No. 2 (AS2) An Audit of Internal Control over Finan-cial Reporting Performed in Conjunction with An Audit of Financial State-ments, was issued by the U.S. Public Company Accounting Oversight Board (PCAOB) to respond to the provisions of Section 404 of the Sarbanes-Oxley legislation as much as possible.

11. As set down in the Approach Paper, IEG might consider commissioning its own testing, if and when: (i) A general random selected testing of controls seems warranted; (ii) Certain controls were found not to have been tested; and (iii) Testing that was done may be deemed inadequate, for example be-cause of sampling deficiencies or other flaws in approach.

12. To discuss entity-level control issues, the IEG team had separate meetings with Bank units dealing with: IDA allocation (FRM); the Integrated Risk Management Framework (SFRSI); issues of fraud and corruption detection (INT); quality assurance (QAG); and safeguards (QACU).

11

2. Management’s Assessment 2.1 This chapter divides into two parts: the first part (paras 2.2-2.11) gives a descriptive synopsis of Management’s approach and method; the second part (from para 2.12) contains IEG’s evaluation of the assessment. Management’s report is in Attachment 1.

Background and Objective 2.2 Management intended to conduct the review of IDA internal controls within the context of the COSO framework, but would focus in this phase of the study only on the assessment of internal controls over compliance. The report states that the COSO framework includes both “top-down” and “bottom-up” analysis. Management determined that Part I of the study would be “more valuable if carried out follow-ing a bottom-up approach,” in order to best track directly the use of IDA resources. Accordingly, the report describes how Management has identified, described and mapped a collection of the key business processes that it used to represent the principal operations activities of IDA, which will be used both to assess compliance in Part I, and to lay a “solid foundation” for the examination of institutional efficiency and effectiveness to be undertaken in Part II. 1

2.3 Management’s objective was to provide an assessment of whether the internal control framework over IDA’s operations pro-vides reasonable assurance to Senior Management and the Board that such operations are carried out in a manner that complies with the provisions of the IDA charter and internal policies governing IDA’s operations, including the mechanisms in place to ensure that funds are disbursed for the intended purposes.

Management’s Method: From the IDA Charter to Policies to Business Processes 2.4 Defining “Compliance” for IDA: Management chose to re-define the meaning of “compliance” in IDA’s case.2 Under COSO, compliance generally implies compliance with local laws and regula-tions. In IDA’s case, as “an international organization established by international treaty with privileges and immunities” Management suggested instead that compliance should be measured against the

Evaluation Essentials The Management

assessment subdivides the compliance controls using business processes

The model requires that controls both be well designed and operate as designed to be effective

IEG finds the method logical, transparent, and convincing and the quality of its results satisfactory

However, it does not capture non-lending activities and may rely on dated OPs and BPs,

The Management assessment identifies several important deficiencies

The bottom-up approach complicates the assessment; affirmative conclusions have to be postponed

The Management assessment aimed to determine whether internal controls, under COSO, provide reasonable assurance that business processes comply with IDA’s charter and policies

CHAPTER 2 MANAGEMENT’S ASSESSMENT

12

relevant provisions of the charter (IDA Articles) and against IDA’s in-ternal policies and procedures.

2.5 Accordingly, Management states in its report that “any com-pliance assessment of internal controls over IDA’s operations must therefore go through a four-step process:

1. identifying key provisions of the IDA Articles that govern IDA’s operations;

2. identifying main policies that were adopted by IDA to ensure that IDA’s operations are carried out consistently with these provisions;

3. identifying the manner in which these policies are intended to be carried out by cataloguing the business processes and key controls put in place to ensure compliance with the identified policies and assessing the “design effectiveness” of these proc-esses and key controls; and

4. assessing compliance with the business process and key con-trols by testing a sample of transactions.” (The subject of Part IB).

2.6 Key Policies and Instruments: Based on this concept of com-pliance, Management sought to establish clear links between the IDA Articles, related policies and procedures, and the actual business processes whose internal controls would be the subject of assessment and testing. The specific hierarchy of these steps is given in summary form in Box 7. It shows how the provisions of the Articles link to spe-cific policies and procedures and the related operational instruments. It shows that the approach stemmed from eight specific provisions of Article V of the IDA Articles, covering allocation and use of IDA re-sources. From over 100 published policies and procedures, Manage-ment made a selection of those that related to the allocation of IDA re-sources and the three key instruments governing IDA operations—country assistance strategies, and the two main forms of lending—Specific Investment Loans (SILs) and Development Policy Loans (DPLs)—citing the four “umbrella” statements in these “flagship” policies and procedures. Having identified these primary operational instruments, Management then also addressed the need to take ac-count of the fiduciary, contractual and safeguards aspects of IDA lending, adding the relevant policy provisions in each of these areas.

Compliance was redefined

IDA Articles were linked to specific policies, procedures, and operational instruments


13

Box 7. Building Blocks in Management’s Approach

IDA Articles Article V—Operations

• Concessional Resources to Less Developed Areas

• Financing High Priority Devel-opment

• Specific Projects and Special Cir-cumstances Lending

• Lender of Last Resort

• Use of Funds for Purposes In-tended

• Due Regard for Economy and Efficiency

• Non-political interference • Linking Disbursements to Ex-

penditures incurred

Policies and Procedures From >100 OPs and BPs Management Focused on

Three Primary Instruments: Country Assistance Strategy(CAS)

Investment Lending Operations (IL) Development Policy Lending (DPL)

“Flagship” OPs and BPs The Flagships Contain four “Umbrella Statements,” namely:

• Umbrella Statement governing financial terms of and eligibility for IDA financ-ing

• Umbrella Statement governing Country Assistance Strategies • Umbrella Statements governing Investment Lending • Umbrella Statement governing Development Policy Lending

Specific Policies for Fiduciary, Contractual and Safeguards Aspects All lending instruments are accompanied by supporting policies and proce-

dures covering:

• Financial management of projects • Disbursement aspects • Procurement aspects • Contractual/Legal and Loan Administration aspects • Safeguard aspects • Quality Assurance

30 Business Process Modules Source: Management Report

2.7 The Business Process Modules (BPMs): Based on this hierar-chy of policies and procedures, Management identified 30 Business Process Modules (BPMs)3 which it saw as representing “the relevant business processes currently in place which staff are expected to use as guidance and best practice when working on IDA operations.” The modules covered the “umbrella” business functions, (allocation, CAS and lending instruments) plus the supporting fiduciary and other as-pects and quality assurance. The material for each BPM included de-scriptions, process flow maps, and specifically defined and located

Thirty Business Process Modules, and their key controls, were identified as representing IDA operations


14

key controls. It is these controls whose design and operating effec-tiveness are the central subject matter of the review. A listing of the BPMs, broadly organized by business function, is given in Box 8.

Box 8. Management’s List of 30 Business Process Modules (Listed by Business Function*)

“UMBRELLA” PROCESSES (8 Modules)

FRM IDA Allocation (IDA Allocation Model) (Post-Conflict Allocation) Debt Sustainability Analysis CAS Products SIL Project Cycle DPL Project Cycle Corporate Review (ROC/OC)

QUALITY ASSURANCE (1 Module)

QAG Processes QEA QSA

FIDUCIARY PROCESSES (21 Modules)

LEGAL SIL Legal Regime DPL Legal Regime Project Changes Contractual Remedies

FINANCIAL MANAGEMENT SIL DPL

PROCUREMENT SIL Procurement Regime Procurement Complaints Procurement Non-Compliance

LOAN ADMINISTRATION Loan Administration SIL Loan Administration DPL Loan Application Review Special Commitment Amendment or Extension Refund Process Loan Cancellation Process Loan Suspension Process Loan Closing (Standard) Loan Closing (Special Procedure)

SAFEGUARDS Safeguards SIL Safeguards Corporate Risk (QACU)

Source: Management listing, organized across business function by IEG.

2.8 Management also explicitly excluded a number of business process modules (10 in total) either because they were deemed not to have direct bearing on lending, or for other reasons, as shown in Box 9. More details on these exclusions are given in Annex E.

Some processes were excluded


15

Box 9. Business Process Modules Excluded from Compliance Assessment Exclusion By Management’s Reso-lution That the Process Does Not Have Critical Bearing on Current Assessment Objective

Exclusion Based on Determination of No Input to IDA Operations

• Country Policy and Institutional Assessment (CPIA)

• Procurement DPL(Procurement is minor in DPLs)

• Post-Conflict Performance Indi-cators (PCPI)

• IEG Process

• Project Preparation Facility (PPF) • IAD Process

• Loan Management—PPF Refi-nancing

• AAA Products

• Annual Report on Portfolio Per-formance (ARPP)

• Inspection Panel Source: Management Methodology Note (working level paper)

2.9 Management has provided a graphic depiction of the full scope of its assessment, show in Figure 2. It depicts the project cycle for SILs and DPLs as the central element, which is linked to the CAS process, to IDA allocation, and to the associated fiduciary, legal, safe-guards and quality assurance processes. These are the essential proc-esses which Management has captured in the 30 BPMs described above, and which Management has taken to represent the totality of IDA operations.

2.10 The Concept of “Design Effectiveness”: While Management has identified the business processes as the vehicles which deliver the various IDA business objectives, it also makes clear that the key con-trols within them (114 in all) are critical to the review of internal con-trols and the forthcoming testing for compliance. As stated in the standards, Management has distinguished between control design and a control operation. To be fully effective, a control must not only be well designed, it must also operate as designed, i.e staff must respect its provisions in the execution of transactions. In Part IA Management has assessed the design effectiveness of these 114 controls under the following definition:

“whether the system of such internal controls is both comprehensive as well as suitably designed to prevent or detect on a timely basis, mate-rial issues of non-compliance or significant control deficiencies.” 4

The model requires that controls both be well designed and operate as designed to be effective

Figu

re 2.

Ove

rview

Sco

pe M

ap o

f Man

agem

ent’s

Ass

essm

ent

Sour

ce: M

anag

emen

t wor

king

mat

eria

ls


17

2.11 Management explained in its report that its working teams conducted their assessment of the design effectiveness of these con-trols through a combination of observation, examination of documen-tary evidence and verification of control design. The output from this process was a series of fully mapped BPMs and accompanying de-scriptive documentation, namely: for each business process module, a Process Overview (PO); and for each key control, a Control Detail Sheet (CDS). An example of a process flow chart—i.e. the graphic mapping of the flow of a transaction through the management sys-tem—together with descriptive materials showing the content of a sample Process Overview and a typical Control Detail Sheet, are also shown in Annex D.

IEG’s Evaluation of Management’s Approach and Method 2.12 The foregoing comprises IEG’s synopsis of Management’s ap-proach. The remainder of Chapter 2 contains IEG’s evaluation of Management’s approach (paragraphs 2.14 -2.26) and of Manage-ment’s main findings (paragraphs 2.27-2.45).

2.13 Management made a number of key choices related to its ap-proach and method, the most important of which are summarized be-low and accompanied by IEG’s findings and conclusions. In particu-lar, choices on basic approach and other scope limitations significantly limit conclusions about the adequacy of internal controls that can be drawn at this stage of management’s work.

2.14 Objectives of the Compliance Assessment: IEG finds the objec-tive of the assessment as stated in para 2.3 above to be reasonably stated, appropriate, clear and complete.

2.15 The Definition of Compliance: IEG agrees that an adaptation of the definition of compliance was necessary in IDA’s case, and finds reasonable Management’s rationale and decision to use compliance with the IDA charter, internal policies and procedures. IEG examined the legal theory underlying this issue and found that very generalized reference to “laws and regulations” does not in the case of a special institution such as IDA provide any guidance as to which laws and regulations are determinate and may give rise to ambiguity as to the role of local law. In the circumstances, it was preferable to refer spe-cifically to the Articles, the lending (or financial) agreements includ-ing the General Conditions, and IDA Policies and Procedures, as con-stituting the governing “laws and regulations” for IDA transactions.1

2.16 From the IDA Articles to Policies to BPMs: Management also correctly located the benchmark for the compliance elements as being the appropriate provisions of the IDA Articles, and the relevant Bank Operational Policies and Bank Procedures (OPs and BPs). Both spe-

The objectives are reasonable, appropriate, clear, and complete

The redefinition of compliance was justified


18

cific Article provisions and the published OPs/BPs were used as the basis to decide which operational instruments and business processes would best represent the panoply of IDA operations. IEG does find the method of developing the 30 business process modules to be logi-cal, transparent, and generally convincing. However, IEG does have comments regarding the possible lack of completeness of the universe of BPMs, as discussed in paragraph 2.18 below.

2.17 Are the Bank’s OPs and BPs an Apt Expression of the Bank’s Policies and Procedures? IEG finds that in each business process module, each control has been linked to one or more specific OPs/BPs, and/or risk statement from the IRMF. However, there are two issues that have been identified regarding OPs/BPs:

The fact that the reform of OP/BP has seriously lagged the pace of change in the Bank Group is acknowledged by Man-agement, and is widely known already;

What was uncovered by IEG during its evaluation, is that there appear to be a significant number of OPs/BPs—some 50%—which were not directly linked by Management to any key controls or business processes. Management has given a satisfactory explanation for those OPs/BPs not linked to spe-cific BPMs, to the effect that: (1) they relate to trust funds and grants, not financed by IDA resources; (2) they apply to other lending products, not SILs; (3) they govern guarantees, which are a very small portion of the IDA portfolio; (4) they govern topics that feed into the processes that were mapped (e.g. eco-nomic evaluation of investment operations; co-financing); or (5) they relate to contractual or other issues that are addressed in the processes that were mapped.

2.18 Does the Cluster of BPMs Adequately Represent the Universe of IDA Operations?2 IEG conducted an analysis of the cluster against some key criteria: What portion of IDA’s operating budget did the clus-ter account for? What product lines? Where processes were excluded from the cluster, did this create gaps in measuring compliance? The content of this analysis is shown in Annex E. IEG concludes that the cluster is broadly representative of IDA’s lending operations (which covers 78% of IDA’s overall operational expenditure). However, by us-ing only SILs to represent all investment lending, Management needs to verify its claim that all other investment loan products have the same controls as SILs (see Annex E paragraph 6). Also, the cluster is essen-tially lending and fiduciary in focus, suggesting that IDA operations are simply IDA lending operations. The cluster omits all Knowledge Products, specifically Analytical and Advisory Activities (AAA), all of which have direct relevance to compliance. As argued in Annex E, AAA has a direct bearing on the quality of IDA lending, as well as ac-counting for about 22% of the IDA budget (almost the same as lending

The method used to develop the business process modules was logical, transparent, and convincing

OPs and BPs may not accurately reflect Bank policies and procedures

The business process modules broadly represent IDA’s lending operations, but not its non-lending activities, and it is not clear that all investment lending uses the same controls as SILs


19

preparation—24%). This is therefore a significant gap in coverage. Management will not be in a position to report on whether internal controls are achieving the business objectives involved until these addi-tional IDA functions and activities are assessed.

2.19 The Quality of Management’s methods in BPM Mapping and Assessing design Effectiveness: IEG conducted a systematic quality evaluation of the methods used in Management’s mapping of the BPMs and assessment of control design. This involved assembling evaluation panels, consisting of 3-4 specialist consultants, who used an evaluation tool created by IEG (the Business Process Template) to give quality ratings to the process of mapping and assessing each of the business process modules. The questions in the Template tested (i) Management’s method for completeness and accuracy of BPM map-ping, and (ii) clarity in Management’s assessment of the effectiveness of their controls design. Details of the content of the Template and rat-ing system are provided in Annex F. Ratings ranged from Highly Sat-isfactory (1), to Satisfactory (2), Satisfactory with Qualification (3) and Less than Satisfactory (4). The evidence on which the ratings were based came from Management’s process maps, the accompanying de-scriptive materials, other working materials Management made avail-able, and the “walkthroughs” i.e. verification interviews between Management and operations staff, many of which IEG panel members attended as observers.

2.20 Rating Results: IEG found that the quality of Management’s mapping and assessment of design effectiveness were of a generally satisfactory standard, but there were also a number of areas with room for improvement. Taking all questions in the Template together, the IEG panels rated 66% of all dimensions as Satisfactory or better (with 93% as Satisfactory with Qualification or better). The average rating across all dimensions was 2.5 (midway between fully satisfactory (rat-ing 2) and satisfactory with qualification (rating 3). More detailed data on the ratings can be found in the Statistical Appendix at Annex G.

2.21 Generally the accuracy and completeness of the process map-ping received high ratings. The principal reason why other ratings were not higher overall was the prevalence, in the descriptive materi-als for some modules, of descriptions of control objectives and specific risks that were more process-oriented than operational in focus.3 In addition, in most descriptions of key controls there was a systematic failure to categorize specific risks by type of risk and to analyze the risk importance in terms of likelihood of occurrence and impact.

2.22 Treatment of Risk: The principal shortcoming affecting the ratings for method and approach was that Management did not at-tempt in any modules to categorize risks by type or priority (this question was rated 4 in all modules). IEG cannot conclude from this

The quality of the mapping and assessment of design effectiveness were generally satisfactory

The main shortcoming was that risks were not categorized by type or priority


20

alone that this failure may have impaired Management’s assessment of control design effectiveness, but prioritizing risks should be an es-sential part of designing internal controls systems, with due regard for economy and efficiency. Also, risks may be more readily priori-tized from an entity-level perspective than within individual business processes. This is an added example of where a top-down approach to the study would have been preferable, and foresees the need for greater attention to risk focus at the organization level.

2.23 Relative Significance and Risk Ranking of BPMs: IEG used its rating process also to rank the modules into groups, according to their strategic importance and risk potential. This was measured by their centrality as a management tool, the magnitude of financial risk in-curred in each process, and the frequency of occurrence. While this involved subjective judgments, it was thought useful as a means of grouping the modules into high, medium and lower risk categories, to separate out the principal modules and those that were more secon-dary. The IEG panels ranked 14 modules in the principal category, 8 into medium risk and 7 lower risk. (See Annex G Table G.7).

2.24 The Overall Approach: Staging the Assessment, Adopting a Bottom-up Approach, Separating Compliance from Efficiency and Ef-fectiveness: 4IEG recognizes the rationale for choosing the bottom up approach—to more directly track the allocation and use of IDA re-sources. IEG also acknowledges the contribution this has made, in the identification and mapping of the relevant processes, which has pro-vided a vision of IDA operations, and a level of detail not hitherto available. The platform of these maps should be very useful going forward both as a source of future tracking of internal controls, and as a means of identifying areas which have scope for streamlining to bring economies and efficiencies. At the same time, the choice of this approach (which runs contrary to standard industry recommenda-tions)5 has created trade-offs which have serious implications for both its Management’s assessment, as well as the IAD review and the pre-sent IEG evaluation, of which IEG would emphasize the following:

Because conclusions on controls within COSO cannot be made piecemeal, but only within the framework as a whole, staging and dividing the study has effectively postponed the ability to make definitive conclusions on the outcomes of each stage of the review until the overall (Part II) assessment has been com-pleted, i.e. until end 2007, or later.

Also the staging of the study between Part IA and Part IB makes conclusions on control design (Part IA) difficult until Part IB has been completed, because final judgments on design effectiveness cannot be made until the operation of the con-trols has also been tested.

IEG panels rated about half the modules to have high risk potential

The approach has involved trade-offs, complicating the overall assessment process


21

Separating compliance and efficiency and effectiveness is really not possible in practice: many business processes and their associated controls are as much to do with compliance as with efficiency and effectiveness, and these are best treated to-gether rather than in sequence. To illustrate, although man-agement has focused its efforts to date on assessing compli-ance, most of the potential issues it has identified are related to efficiency and effectiveness.

Other scope limitations flowing from the delineation of the study—in particular the decision to deal with IT systems and field offices in Part II, have yet further limited the conclusions that can be drawn in Part I, especially given IDA’s increasing decentralization, and the growing importance of IT in main-taining the integrity of central controls.

In principle, it is possible that by completing the entity-level review during Part II, and addressing the postponed parts of the framework, Management will be able to mitigate these de-ficiencies in approach by linking results from the various parts together, to provide an overall statement. However, this will depend on there being no changes in any basic parameters: controls will be assessed at different points in time, and poli-cies, procedures, systems, organization structures may change during this period.

2.25 A Note on Scope Limitations: It was stated early in this report that IEG would be evaluating the implications of the various scope limitations that Management, by intent, or by implication, has im-posed on this first part of the study: the separation of COSO objec-tives, the staging of the parts of the study, the postponement of the treatment of entity-level controls, and the examination of IT systems and decentralized locations. The impact these limitations have had on the outcome of the assessment and evaluation so far have been item-ized in much of the foregoing analysis. No separate evaluation of the implications of these limitations is therefore needed here.

2.26 Summary Evaluation of Management’s Methods: In summary, IEG recognizes the underlying reasons why Management decided to adopt a process-based approach, to divide the study into stages, and to impose other scope limitations. However, in considering the trade–offs that this gave rise to, IEG comes to a mixed conclusion on the ap-proach overall, finding both strong points and some problem areas:

Strong Points: IEG finds that the following key elements of the approach have all contributed positively to the assessment: the definition of the objective of the assessment; the re-definition of compliance as it applies to IDA; the identification of the relevant provisions of the IDA Articles; and the related links to policies and procedures, as the basis to build a cluster of rep-


22

resentative business process modules; and the actual mapping and description of the modules, which formed the bases for testing.

Problem Areas: IEG observes several issues: issues with the quality of OPs/BPs; that the chosen cluster of BPMs represents well IDA lending products, but excludes AAA and other Knowledge Products—a significant omission; the combination of factors stemming from the bottom-up, three-stage division of the study over time, with a separation of COSO elements. All of the latter oblige the postponment of definitive conclu-sions on the earlier stages of the study, until the overall review has been completed.

Management’s Main Findings and Conclusions: IEG Comment and Evaluation 2.27 The main findings from Management’s Part IA assessment, in terms of specific issues relating to controls design and operation, will be discussed under three headings:

Management’s broad conclusions on what it has achieved in the review so far;

The main highlighted deficiencies, which Management sees as the most important weaknesses so far revealed;

A listing of a number of additional issues, which Management regards as notable but less significant. The materiality of these is still to be established by Management, and many may not rise to the level of significant deficiencies or material weak-nesses under the agreed audit standards described in Annex B.

2.28 What follows is a summary of these main findings, and con-clusions, accompanied by a comment and evaluation from IEG, case by case.

MANAGEMENT’S BROAD CONCLUSIONS 2.29 Management found that its assessment approach, unprece-dented in the Bank Group,6 was such as to “capture a representative pic-ture of the control environment over IDA’s operations at the transaction level” and that it had contributed information which would be useful for strategic and rationalization decisions regarding IDA operations going forward. The examination of the key policies and procedures that govern IDA operations and the mapping and review of the key business processes and associated controls that enable compliance with these policies confirmed in its view that the processes and con-trols that apply to the four primary instruments for carrying out IDA operations (i.e. Allocation mechanism, CAS, IL and DPL) have been designed to verify that scarce IDA resources are made available to

Management claims its approach broadly captures IDA processes


23

support priority development activities in the poorest eligible mem-ber countries.

2.30 IEG Comment: IEG states that it cannot at this stage make an affirmative statement regarding IDA controls over allocations and use of funds, but it agrees that the transaction-level mapping of business processes and the definition of key controls has contributed valuable information not hitherto available and it agrees that this could serve IDA well going forward, by providing a useful platform for future control tracking and reviews, and also for identifying areas where streamlining may be possible.

MANAGEMENT’S HIGHLIGHTED DEFICIENCIES (PARAGRAPH 26 OF ITS REPORT)

2.31 Deficiency 1: Management stated in its Report that it “had diffi-culties with obtaining timely access to relevant documents that are needed to carry out the compliance testing portion of the assessment” and it went on to explain that while external auditors confirm that many institutions transiting from manual to electronic filing systems have similar reten-tion and accessibility problems, Management regards this as a serious issue, and has consequently commissioned a Task Force to address this issue and present solutions within six months.

2.32 IEG Comment: Document retrieval has clearly become a ma-jor issue, and is an area of potential material weakness.7 For now, IEG takes note of Management’s straightforward recognition of the issue, its diagnosis, and the suggested remedial action plan. IEG suggests that this problem may have broader roots than Management has de-scribed. No doubt the transition to an electronic Bank is a (possibly the) central issue; but this has also been accompanied in the past dec-ade by an ensuing cultural shift in the roles of staff (e.g. in the interac-tion between Task Team Leaders (TTLs) and support staff), as well as by the major decentralization of the Bank in the past decade. Since the present Part I stage of the study has explicitly excluded examination of IT and field offices, this, also, could have affected these results. Fi-nally, while Management’s bottom-up approach successfully demon-strated the severity of this problem, a top-down approach might have given the whole exercise more visibility with line managers and this could possibly have strengthened unit cooperation and facilitated the rigorous process-level demand for documentation, that is required for the process-based method.

2.33 It is too early at this stage of the review to make definitive con-clusions, but IEG believes that the document retention issue is an area of potential material weakness, and takes note of the fact that Man-agement has already mounted a high-level remedial effort.

Document retrieval is a major issue


24

2.34 Deficiency 2: Management has identified the fact “that the OPs and BPs included in the Operational Manual are not keeping pace with the changes on the ground that are being introduced from time to time.” It gave as examples BP10.00 on processing investment lending from identifi-cation to Board approval, which is in need of urgent update, the 12.00 series governing disbursements, and OP/BP10.02 on financial man-agement. Management goes on to state that “As part of Management’s assessment of the effectiveness and efficiency of IDA’s internal control framework, Management intends to look at the current processes underlying the policy revision to determine if they need to be revised to facilitate more efficient and timely updating of operational policies and procedures.”

2.35 Deficiency 3: Management states that “the policy framework gov-erning IL operations is too complex and disjointed, making it hard for staff to identify all the policies with which they are expected to comply when working on IL operations” and it explains that it is actively working on rational-izing and consolidating the OPs/BPs governing investment lending.

2.36 Deficiency 4: Management states that its assessment showed that many staff find the existing processes and documentary require-ments very onerous and inefficient, and that “Management intends to issue in the next few months standard updated operational templates to be used by staff in documenting various steps in the IL and DPL processes.”

2.37 IEG Comment: IEG observes that there are issues relating to overall status and quality of the OPs, BPs and processes governing pro-ject preparation. IEG also observes that this issue is not new, and has been referred to in earlier IEG studies.8 From the perspective of internal controls, the issue may be more serious than Management has stated. If the quality of the OPs/BPs is poor, in terms of the fact they have not all been amended to take account of change in the Bank Group, this could call into question whether they provide a valid benchmark for compli-ance testing. Other agencies facing a similar situation have sometimes decided to postpone compliance testing until the policies have been re-formed. In the case of IDA, Management faces a decision: whether to proceed with the assessment with the present OPs/BPs, and perhaps settle for a qualified assertion at the end, or whether to postpone the as-sessment of compliance, until the OPs/BPs have been brought current. IEG believes a postponement may be impractical, and argues elsewhere (see paragraph 4.5 third bullet) that completion of Part II needs to be timely.

2.38 While it is too early in the review to make definitive conclusions, IEG believes that the status of the OPs/BPs is a second area of potential material weakness but notes that Management has stated that it has a reform program, consisting of both streamlining and updating components. IEG believes this program should be treated as a high priority, even accelerated.9

OPs and BPs are not keeping pace

OPs and BPs are too complex and cumbersome


25

2.39 Deficiency 5: Management stated that its assessment had found a “disparity in the frequency of corporate reviews of SIL and DPL operations, with all DPLs being subject to such review and relatively few SILs. ... Man-agement is examining whether there is a need to review criteria for submitting investment lending operations that raise special risks or issues to the corporate review process.”

2.40 IEG Comment: IEG agrees that management should review its criteria and make revisions as necessary to ensure operations with special risks are subject to corporate review.

MANAGEMENT’S LIST OF ADDITIONAL ISSUES 2.41 In addition to the five high-level findings identified in Man-agement’s report and discussed above, Management provided IEG with a list (not appended to its report) of additional potential issues relating to internal control design effectiveness. The issues require additional work to substantiate the findings and determine their sig-nificance and the remedial actions, if any, that may be required to mitigate risks. This work is to be made as part of the follow-on Part 1B stage focusing on the operational effectiveness of the controls. At that time, IEG anticipates that Management will determine which of these issues, or combination of issues, may rise to the level of a deficiency, significant deficiency, or material weakness, as defined by the agreed standards described in Annex B.

2.42 In summarizing its detailed list of potential control issues, Management identified areas that merit a closer look in its follow-on work, as follows:

“Several existing policies and procedures need to be updated or enhanced or, in some instances, additional guidance needs to be introduced.

Certain system capabilities or system-related controls need to be better aligned with the process requirements.

Timeliness of processes related to managing individual credits should be improved.

Variances in regional implementation of institutionally en-dorsed guidelines need to be reviewed to ascertain whether these variances are appropriate.

Certain procurement processes and controls require enhance-ments to strengthen effectiveness.

Processes surrounding SILs with regard to project changes or contractual remedies need to be strengthened.

Clarification should be issued to require staff, which provide conditional clearances or feedback on project documents, follow up to ensure that their comments have been incorporated, as ap-propriate.”

Other issues were identified but their materiality has not yet been established


26

2.43 Annex C provides additional information on internal control issues identified to date, including specific examples to illustrate the character of the (i) several potential internal control issues or deficien-cies identified by Management, (ii) additional issues identified by IEG as potential internal control deficiencies; and (iii) noted deficiencies identified by IEG in the materials submitted by Management as part of its Part IA assessment.

Findings from IEG’s Analysis 2.44 IEG conducted its own independent analysis of Management’s assessment, both by examining Management’s materials and by at-tending selected “walkthroughs” as observers. The analysis revealed a number of additional issues to those that Management had identi-fied, some of which related to controls issues, others to deficiencies in Management’s materials, mapping and descriptions. In both cases, more details are given in Annex C:

ISSUES RELATED TO CONTROLS In the modules dealing with disbursements and loan amendments

(Modules 19 and 21) IEG found that Task Team Leaders are af-forded much flexibility in the OPs that govern IDA, which per-mit them to make changes in a project disbursement category amount, on the understanding this will be documented later, but IEG could not find a control which ensured this would take place. Also, IEG could find no mechanism whereby a TTL could inform the Loan Department to place a hold on disbursements, if she/he had uncertainty that funds were being disbursed for the purposes intended.

In the refunds and loan closing modules (Modules 22, 25, 26) IEG learned that the Loan Department has no mechanism to en-sure that balances remaining in special accounts (greater than 1% of loan amount) and due for refund to IDA are eventually received.

In the safeguards module for a SIL (Module 28) IEG found that, if certain safeguard documents to be prepared by the country in the preparation phase are inadequate, there is no mechanism in the process for it to be sent back to be redone.

In the procurement complaints module (Module 15) IEG found that there is no control to ensure that all complaints are en-tered into the complaints database. Also, there is no control to ensure that reports from the complaints database are followed up on and that all complaints are handled appropriately.

2.45 Summary Evaluation of Revealed Deficiencies: In IEG’s opin-ion, these issues taken individually are deficiencies. Whether, indi-vidually or in combination, they would rise to the level of significant

IEG has identified some specific deficiencies in controls


27

deficiencies or material weaknesses would best be judged as part of the totality of the revealed deficiencies, including those uncovered by both Management and IAD, and this should be deferred until at least the completion of the controls testing to be done in Part IB. Failure to establish the materiality of these deficiencies by then would interfere with the ability to make affirmative conclusions on the effectiveness of the overall internal controls framework.

ISSUES RELATED TO MANAGEMENT’S DESCRIPTIVE MATERIALS AND MAPPING 2.46 IEG also noted certain imperfections in Management’s de-scriptive materials, of which the following are highlight examples (more details are given in Annex C):

In the description of specific risks linked to key controls, Man-agement did not categorize these risks as to type (e.g., finan-cial, operational, or reputational) or analyze the risks in terms of likelihood of occurrence or impact; (para 2.21)

The DPL module did not address the potential impact on IDA’s reputation and the country if the DPL objectives are not accomplished;

Loan management modules did not mention the area of re-payment and amortization risks;

In some BPM process maps, the sections of the main lending processes subject to QAG assessments (Quality of Supervision Assessment; Quality at Entry Assessment) had no links to show where these QAG interventions would be made.

The CAS module omitted to show links to the portfo-lio/pipeline review, to AAA, including CEM and PRSP, and did not show the inputs of the Sector Networks;

2.47 These comments complete IEG’s evaluation of Management’s assessment of the present stage. Chapter 3 deals with the IAD review, while Chapter 4 presents IEG conclusions and recommendations.

1. Quotes from page 2 of Management’s Report.

2. Ibid page 2.

3. At the time of completing this report, Module 30, on Debt Sustainability Analysis, was still being completed, so the assessment and IEG evaluation were effectively conducted on 29 modules. IEG assumes that Module 30 re-mains part of the universe going forward. Further, Management regards the IDA Allocation Model and the Post Conflict Allocation as sub-processes to the main Allocation module, so by some counts Management refers to 27 modules.

4. Management Report, paragraph 21.

1. The Bank and IDA are affiliated multi-national development agencies each with its own jurisprudential personality established through international treaty by Articles of Agreement between the member countries. Article VIII

And there are some imperfections in Management’s descriptive materials

NOTES


28

Section 10 of IDA’s Articles of Agreement requires that “Each member take action necessary in its own territories for the purpose of making effective in terms of its own law the principles set forth in the Articles of Agreement and shall inform the Association of the detailed action which it has taken.” This principle is carried forward from the Articles into the IDA lending agree-ments (which incorporate the General Conditions) and which stipulate that in case of conflict the member/borrower cannot raise the local law as a justi-fication for failure to implement the conditions set forth in the lending agreements. It is not the legal duty of IDA to monitor the compliance by the borrower with all its own laws and regulations. However, IDA has to be as-sured that the conditions it needs for a successful project are spelled out in the lending agreements and will be implemented notwithstanding conflict with local law. For this purpose, IDA requires (as a standard condition of ef-fectiveness) a legal opinion satisfactory to IDA confirming that the provi-sions of the lending agreement are valid and binding on the borrower.

2. Management comment: Management’s decision to focus this exercise on IDA lending products and not specifically focus on AAA and other Knowl-edge Products was consistent with the main objective of assessing the inter-nal controls in place for ensuring how borrowers use IDA resources for the purposes intended. The fact that a large percentage of administrative budget is spent on AAA does not in and of itself suggest the connection between AAA and lending where IDA controls under review would apply.

3. Take, for example, Module 9, Contractual Remedies. The Process Over-view describes the objective of the process as: “Ensure contractual remedies are applied when necessary…..” Though not wrong, it sounds as if the objec-tive of the process is the process, and the explanation carries little of the op-erationally substantive facts that: IDA assistance is always contingent on ful-fillment of project agreements and other covenants. This implies a need for sanction and remedies where agreements are broken, or covenants not met….etc, so that when the occasion arises, there is a structured process to seek remedies and sanctions….etc., and this is the purpose of Module 9.

4. Management comment: IEG’s criticism of the Management’s approach of dividing the assessment into two parts and focusing the first part on transac-tion-level controls has to be taken in the context of the unprecedented nature of this assessment, the intensity and scope of work required as well as its primary objective of assessing the existing controls for ensuring that IDA funds are used for the purposes intended. As has been discussed with CODE and the Audit Committee in November 2005 and July 2006, the review of IDA’s Internal Controls, to date, has required a massive amount of work on Management side, involving the identification and categorization of the rele-vant policies and a rigorous review of 30 business processes that are key to IDA operations and resulting in 700 pages of documentation. We believe that everyone ( IEG, IAD and Management) is in agreement that the work per-formed represents a landmark step in the understanding and evaluation of IDA’s control framework and is unique in the development community. The work performed to date is particularly useful precisely because of the meth-odology utilized. In recognition of the practical difficulties of undertaking one comprehensive study, Management has in effect identified three phases: in the first (Part IA), which is the subject of this report, Management has sought to ensure that procedures embed key controls to assure compliance with policies; in the second (Part IB), Management will assess whether these procedures are complied with in practice; and in the third (Part II), Manage-ment will assess whether the mechanisms in place to monitor that IDA op-


29

erations are efficient and effective are adequate. Management believes that this methodology allows for a systematic and manageable analysis, and the development of specific and actionable findings. While Management agrees with IEG that the methodology applied makes it difficult for IEG to provide an overall definitive conclusion at this stage, given the trade-offs, in a re-source constrained environment and real time-limitations on what could be done, the approach followed proved to be preferable as the findings, and re-lated action plans are likely to be the major themes of the entire assessment. In this regard, it is important to note that IEG has confirmed that “The mapped Business Process Modules have provided a concrete and transparent means of identifying, assessing and testing key controls” (para 4.3 (ii), em-phasis supplied) and found that “Management’s approach has yielded con-crete results in revealing deficiencies and potential weaknesses” (para 4.3 (v)). Management is committed to complete the next phases of this exercise, namely Part IB relating to assessment of operation effectiveness of the con-trols identified and Part II relating to the overall efficiency and effectiveness. However, it is important to note that while these parts are expected to refine Management’s findings and conclusions, given the nature of conclusions to date and the rigor of the work performed, it is reasonable to expect that the nature of the conclusions under Part IB and II will be the same as the impor-tant findings and conclusions which resulted from the work carried out un-der Part IA.

5. An entity-level review would have allowed a prior assessment of how well the COSO framework is being observed as a whole, it might have suggested priority areas of risk, and it would have informed the operating units in the Bank of the impending review, and given them time to prepare for the busi-ness process tests. IEG can find few if any precedents for major internal con-trol reviews which have not started at the entity level. AS2 (see annex 2) states that “it may be appropriate for the auditor to test and evaluate the de-sign effectiveness of company-level controls first.” The Policy Statement is-sued by the PCAOB after the first year of experience with AS2 states that “this Policy Statement expresses the Board’s view that, to properly plan and perform an effective audit under Auditing Standard No. 2, auditors should … use a top-down approach and that “Auditing Standard No. 2 was de-signed to be applied from the top down.”

6. Management’s statement that this type of review is without precedent is true for the Bank and IDA, but there are some precedents for this in other agencies. The United States federal government agencies have been making such assessments under the Federal Managers’ Financial Integrity Act (FMFIA) since its passage in 1982. For example, the United States Department of the Treasury, in its Fiscal Year 2005 Performance and Accountability Report (page 23), stated that “As a result of our evaluations, Treasury can provide rea-sonable assurance that the objectives of the Federal Managers’ Financial Integ-rity Act have been achieved, except for the remaining material weaknesses…” (which were listed). Treasury went on to list the management control objec-tives under FMFIA among which included “to ensure that programs achieve their intended results, resources are used consistent with overall mission, pro-grams and resources are free from waste, fraud and mismanagement, and laws and regulations are followed.”

7. Management comment: The IEG report states that IEG regards the issues identified by Management relating to documents retention as a “potential material weakness.” Management questions the validity of IEG’s use of the term “potential material weakness” given the speculative nature associated


30

with any “potential” material weaknesses. The outcome of any prospective work is by definition unknown. Therefore, Management believes that it would be more prudent at this stage not to guess or prejudge what the pos-sible outcome might be but rather limit any conclusions to the factual state-ments and specify that as part of the follow on phases this issue would be looked at so as to assess both the seriousness of any problem and how best to address it.

8. See, for example, the AROE for 2002.

9. Management comment: The IEG report states that IEG regards the status of some OPs and BPs as “potential material weakness.” As with the issues relating to document retention, Management questions the validity of IEG’s use of the term “potential material weakness” relating to OP/BP status given the speculative nature associated with any “potential” material weaknesses. In addition, Management believes that while the current status of some OPs and BPs may have a negative impact on the efficiency and transaction costs of processes and key controls associated with such OPs and BPs, it does not constitute an actual or potential material weakness relating to compliance. In this regard, it is also important to note that the policy reform effort under-way as part of the modernization agenda has been focused on simplifying and streamlining the existing policies and procedures, not in addressing weaknesses or gaps in the current statements. Most notable examples of these types of policy reforms are the reform of OP/BP 8.60, governing De-velopment Policy Lending, reform of policy on expenditure eligibility (OP/BP 6.00), and the policy on additional financing for investment lending (OP/BP 13.20). Recognizing this, as part of the modernization agenda, Man-agement discussed with the Board that other updates of policies relating to investment lending would proceed only after progress was made on mod-ernizing non-policy aspects of investment lending and not in parallel. It also has been long recognized by Management and the Board that the current processes for preparation, review, consultation, approval and issuance of re-vised policies are extremely time and resource consuming, often resulting in a lag between the time when a need for a given policy update is identified and the final approval and issuance of the revised or updated policy. To re-flect this, and ensure that this issue is properly addressed as part of this exer-cise, in paragraph 26 C of its Findings and Recommendations, Management suggests that “As part of Management’s assessment of the effectiveness and efficiency of IDA’s internal controls framework, Management intends to look at the current processes underlying policy revision to determine if they need to be revised to facilitate a more efficient and timely updating of operational policies and procedures.” Management believes that this is indeed an impor-tant efficiency issue that should be addressed under Part II of this exercise.

31

3. The IAD Review and Report

Context for IEG’s Review of IAD’s Work 3.1 This chapter contains a description of IAD’s objectives, scope and approach to its review (paragraphs 3.2-3.3), and a summary (paragraphs 3.4 and 3.5) of the main observations that IAD has re-corded in its report. In the latter section, where appropriate, IEG comments on and evaluates IAD’s main findings. 1 The complete IAD report is in Attachment II.

IAD’s Objective 3.2 IAD’s Terms of Reference2 stated as its objective in reviewing Management’s assessment of internal controls to express an opinion on whether the assessment of internal controls over IDA operations, relating to their compliance with the IDA charter and its internal po-lices and procedures, has been fairly stated, based on the criteria es-tablished in the COSO framework. As a result of Management’s deci-sion to divide its assessment into Part I—internal controls over compliance—and Part II—internal controls over operational effec-tiveness and efficiency—and subsequently to divide Part I into Parts IA and IB, IAD had to develop a more specific scope and approach for Part IA. IAD has also concluded that its opinion would have to await completion of Part IB.

IAD’s Scope and Approach for Part IA 3.3 Similar to IEG, IAD noted a number of scope limitations in Management’s work, including the deferral of entity-level controls and internal controls over efficiency and effectiveness of operations, processes excluded by Management, and deferral of the evaluation of information technology controls. Using relevant concepts from AS2 tailored for compliance and operational controls, IAD performed the following work.

Process Documentation: IAD reviewed high level process flowcharts, descriptive materials of processes and control ob-jectives, risks and key controls provided by Management. IAD

Evaluation Essentials The IAD review identified

many of the same issues identified by IEG

IAD questions exclusion of some processes

Management has not covered fraud and corruption

IAD finds the Management approach contrary to recommended industry practice

Conclusions await completion of Part IB

The IAD review objective was to express an opinion about whether the Management assessment was fairly stated and based on COSO criteria

CHAPTER 3 THE IAD REVIEW AND REPORT

32

reviewed whether key controls identified by management ap-peared adequate to satisfy control objectives, and identified potentially missing key controls.

Workshops/Review Sessions:3 IAD observed work-shops/review sessions conducted by Management with sub-ject matter experts to review the design effectiveness of key controls as identified and documented by Management, chal-lenging, seeking clarification and identifying potential defi-ciencies as appropriate.

Revised Process Documentation: IAD reviewed revised proc-ess descriptions incorporating changes identified in the work-shops/review sessions, and potential control issues identified by Management.

Deficiency Tracker: IAD provided Management with a list of 59, potential deficiencies in documentation and/or design identified during its review, four of which also were included in Management’s list of potential deficiencies.

Process Walkthroughs: IAD attended process walkthrough sessions convened by Management with operating personnel responsible for three processes (Country Assistance Strategy, Investment Lending and Development Policy Lending) to con-firm the operation of process controls (for at least one transac-tion for each process) from inception to completion, as docu-mented in the workshops/review sessions.

Management’s Report: IAD reviewed drafts of Management’s report and provided comments as appropriate.

IAD’s General Observation and Key Issues 3.4 General observation: Overall, IEG and IAD raised many of the same issues as a result of their work. IAD commented positively on the comprehensive and unique nature of Management’s assessment within the multilateral development banking community and stated that it could provide a compelling baseline for identifying opportuni-ties to streamline operations and internal controls and improve effi-ciency and consistency.

3.5 Key Issues: IAD identified eight key issues during its review of Part IA of Management’s assessment. Each of the key issues is summarized briefly below along with IEG’s observations.

IDA processes selected: IAD raised questions about Manage-ment’s decision to exclude certain processes 4in determining compliance with fiduciary aspects of IDA’s lending opera-tions, as well as limiting its scope to processes applicable to SILs and DPLs. IEG’s evaluation also raised a number of con-cerns about IDA processes excluded from Management’s as-

IAD and IEG raised many of the same issues

IAD questions exclusion of some processes


33

sessment (particularly knowledge products) and using SILs alone to represent all investment lending. (para 2.8 and Annex E, paras 7-9)

Information technology (IT) controls: IAD commented on Management’s plan to defer assessment of IT controls to Part II as part of assessing entity-level controls and took the posi-tion that key IT controls need to be assessed in Part IB to reach reliable conclusions. IEG’s work also identified the deferral of IT controls to Part II as a significant scope limitation, which limits conclusions that can be drawn from Management’s as-sessment on Part I. (2.24)

Fraud and corruption controls: IAD commented that Manage-ment had not identified and documented controls focused on mitigating risks associated with fraud and corruption at the process level. IAD also noted that Management had not as-sessed the adequacy of other controls to satisfy such objectives, such as ensuring that control implications identified during fraud and corruption investigations are adequately addressed. In IEG’s view, controls over fraud and corruption should first be addressed at the entity-wide level, which would include con-trols to assure that issues identified during fraud and corrup-tion investigations are adequately addressed. Work on fraud and corruption controls at the entity-wide level could then in-form the need to assess key controls at the process level impor-tant to preventing or detecting fraud and corruption. IEG also observes that the controls assessments completed by Manage-ment implicitly also dealt with fraud and corruption prevention issues, but agrees with IAD that this could have been made more explicit.

Outdated OPs and BPs: IAD notes “absent processes to en-sure that policies are current, controls to ensure compliance with such policies would not be meaningful, even if current practices meet business needs.” IAD recognizes that Manage-ment has committed to review the process for updating the OPs and BPs and to examine the appropriateness of regional variances in implementation. IEG’s work identifies this as a significant issue raised by Management’s assessment that IEG believes is an area of potential material weakness, and rec-ommends that Management’s reform work be treated as a pri-ority.

Categorization and Remediation of Deficiencies: IAD states that Management’s assessment in Part IA has yet to determine whether identified deficiencies pose, in the aggregate if not individually, significant or material risks to the attainment of control objectives. IEG’s work also identified this issue and IEG agrees with IAD on the need for Management to evaluate in Part IB the significance of the deficiencies identified by

Fraud and corruption are not covered by Management

OPs and BPs are outdated

Management needs to establish materiality of deficiencies


34

Management’s assessment and the additional deficiencies identified by IAD and IEG (para 2.45).

Document Retention and Accessibility: IAD takes note of Management’s significant difficulties in obtaining timely ac-cess to relevant documents for compliance testing and con-cludes that in IAD’s experience this represents a significant control design deficiency. While Management has agreed to address this issue, IEG concludes that this is clearly a major is-sue that could reach the level of material weakness. In particu-lar, IEG notes the potential relationship between the document retention issue and the major decentralization of the Bank in the past decade and the role of information technology in document retention (para 2.32).

Assessment of Entity-level Controls: IAD comments on the de-cision to defer the assessment of entity-level controls to Part II and notes that Management’s conclusions on control effective-ness as a result of Part I will need to be reconsidered once en-tity-level controls have been examined. IEG identifies the deci-sion to delay assessment of entity-level controls to Part II as a major scope limitation and finds that Management’s decision to employ a bottom-up approach to assess controls as contrasted to a top-down approach, starting with entity-level controls, runs contrary to standard industry recommendations.

Walkthrough of Process Documentation: IAD states that “Management’s assessment of design effectiveness of internal controls under Part IA of the review included walkthroughs of process documentation for three of the 29 documented in-scope processes (CAS, SILs and DPLs) from inception to com-pletion.” IEG disagrees with IAD’s definition of walkthrough, in this context, and regards most of the 29 in-scope processes to have been subjected by Management to a walkthrough process essentially similar (as modified to suit the nature of IDA operations) to that described in the AS2 text summary of the concept.

Completion of Remaining Stages: IAD “strongly recommends reconsidering the relative cost-benefit of continuing immedi-ately with the remaining issues of the assessment versus con-tinuing after addressing significant deficiencies identified in Part IA” (PAGE 7, IAD report). IEG has considered this option, but regards a postponement as unnecessary and impractical and believes, on the contrary (as argued in paras 2.24 and 4.7), that the remaining stages should be completed expeditiously.

1. IEG had a very limited period of time to review IAD’s report, so these ob-servations are to be taken as reflecting IEG’s evaluation at this stage of its analysis. IEG will be broadening its evaluation in the light of further work to be completed during Part IB and beyond.

Document retention and accessibility are major issues

Management approach is contrary to industry recommenda-tions

NOTES


35

2. Terms of Reference for a Review of Management’s Assessment of Internal Controls over IDA Operations, IAD, May 16, 2006, issued as a memorandum to the Vice Presidents of CTR and OPCS, which IAD also sent to AC and CODE.

3. IAD and Management have used different applications of the term “Walk-through.” They also have differing views on the extent of work to be completed under this process. IEG has used the term (as described on page iv) to signify the process by which Management has verified process mapping and design effec-tiveness, without denoting acceptable quality of the work completed.

4. Economic and Sector Work (ESW), Report on Observance of Standards (ROSC) Independent Evaluation Group (IEG) and Internal Audit Depart-ment (IAD) processes, among others. (see Box 6 on page 15).

37

4. Conclusions and Recommendations

4.1 This final chapter presents IEG’s main conclusions from its evaluation of the work completed so far. The chapter provides a broad overview, a summary of the concluding evaluation of both Management’s assessment and the IAD review, and it concludes with a set of IEG recommendations going forward.

Overall IEG Evaluation 4.2 IEG recognizes the reasons why Management adopted the process-based and phased approach, and acknowledges its merits. But considering also the scope limitations this necessarily implied, IEG ar-rives at a mixed conclusion on the completion of this first stage of the study: At the Transactions Level: robust progress has been made in de-fining, locating and assessing key internal controls, and the results from this approach have revealed a number of deficiencies and possible weaknesses in the underlying controls; At the Controls Framework Level the general approach and scope limitations applying to this stage of the assessment prevent positive assertions being made now regard-ing the effective operation of the overall system of controls.

4.3 At the transactions level progress can be measured in the fol-lowing ways:

As a basis to test for compliance, Management has made a credible linkage between the IDA Articles, the Bank’s policies and procedures, and the business processes identified to rep-resent IDA operations;

The mapped Business Process Modules have provided a con-crete and transparent means of identifying, assessing and test-ing key controls;

Management’s methods of mapping and assessing the BPMs have been of a generally satisfactory quality, though with some notable qualifications relating to the treatment of risk, and the need to improve some of the descriptive materials.

Management’s “walkthrough” method of verifying the accu-racy of the selected business processes and testing the design effectiveness of their key controls was rigorous, comprehen-sive, transparent, and documented to a largely satisfactory

CHAPTER 4 CONCLUSIONS AND RECOMMENDATIONS

38

standard, consistent with general concepts of AS2; (some fur-ther walkthroughs may still be needed);

Management’s approach has yielded concrete results in re-vealing deficiencies and potential weaknesses: Management uncovered two deficiencies (which IEG regards as potential material weaknesses—document retention; imprecise and un-reformed OPs/BPs), as well as one other potential significant deficiency; in addition, Management compiled a list of some other issues whose materiality is to be examined and assessed in the next stages of the study. Also, on the basis of their re-view of Management’s work, both IAD and IEG have added to this list, with IAD identifying 55 issues (35 documentation is-sues; 20 potential control deficiencies), and IEG identifying four control deficiencies, and six areas of imperfection in the descriptive materials and process maps.

4.4 At the Level of the Controls Framework, the weaknesses stemming from the inevitable trade-offs and scope limitations inher-ent in Management’s chosen approach can be summarized as follows:

Conclusions on controls within COSO cannot be made piece-meal, so staging and dividing the study has effectively post-poned the ability to make definitive conclusions on the out-comes of each stage of the review until the overall (Part II) assessment has been completed, i.e. until end 2007, or later.

Even the staging of the study between Part IA and Part IB makes conclusions on control design (Part IA) difficult until Part IB has been completed.

Separating compliance and efficiency and effectiveness is really not possible in practice: many business processes and their associated controls are as much to do with compliance as with efficiency and effectiveness, and these are best treated to-gether rather than in sequence.

Other scope limitations flowing from the delineation of the study—in particular the decision to deal with IT systems and field offices in Part II, have yet further limited the conclusions that can be drawn in Part I.

4.5 By completing the entity-level review during Part II, and ad-dressing the postponed parts of the framework, Management should be able to mitigate these deficiencies in approach by linking results from the various parts together, to provide an overall statement. However, this will depend on there being no changes in any basic pa-rameters: controls will be assessed at different points in time, and policies, procedures, systems, organization structures may change during this period.


39

4.6 Summary of Key Observations by the Reviewing Parties: This three-phase assessment, review and evaluation has given rise to a complex combination of descriptions of method, summary of find-ings, and evaluative observations. As an aid to provide a simplified summary of the main issues that have been identified by Manage-ment, IAD and IEG, Box 7 below provides a tabulation of issues. It separates issues relating to approach and method from those relating to the results of Management’s assessment at the conclusion of Part IA. It shows that there is a considerable commonality between IEG’s findings and the observations that IAD has made on issues raised by Management’s approach and method. It also shows areas in which IEG has made observations where IAD has not commented, (e.g. process issues; quality of Management’s BPM mapping) and cases of disagreement (e.g. over the definition of walkthrough). IAD has, through the walkthrough process been able to identify 55 additional issues (documentation and control issues) to be further examined for their nature and materiality. Taken overall, IEG finds the coverage of issues for this stage of the review to be robust and wide-ranging.

Box 10. Summary of Principal Issues Identified by Management, IAD, and IEG

Issues relating to approach and method

Management IAD IEG

A: Framework Issues 1. Bottom-up versus Top-down



2. Staging and Dividing the Assessment



3. Dealing only partially with COSO components

- Postpones Conclusions

4. Scope Limitations IT to be assessed in

Part IB

Optional; IT is part of Entity Level controls

B: Process Level Issues: 1. Definition of Objectives, Compliance

- Acceptable

2. From Articles to Key Policies and Procedures

- Acceptable

3. Linking OPs/BPs Explanations offered

- Only 50% linked to BPMs

4. Identifying BPMs - Acceptable 5. Quality of BPM mapping - Satisfactory,

some qualifications

5. The Cluster as Representing IDA Operations

Issue: Excluded Processes

a. Lending: Test ILs

b. Excluded AAA/KP

Continued


40

Box 10 (continued) Issues relating to results: major

controls issues

Highlighted Controls Issues By Management

1. Document Retention and Accessibility



Potential Material

Weakness 2. Current Status of OPs/BPs: a. OPs/BPs outdated, often not current b. Complex, disjointed policy framework c. Onerous, inefficient processes



Potential Material

Weakness

3. Disparity in Corporate Review SILs and DPLs.

Highlighted -- Highlighted

By IAD (3) 1. Outdated OPs/BPs

Highlighted Highlighted

2. Definition of Walkthrough Disputed Management Consistent with AS2 concepts

3. Fraud and Corruption Controls

Should be assessed at

process level

Start with Entity level

controls; could have been

more explicitly treated

By IEG (4) (i) No control over “subject to” disbursement changes; (ii) no assurance all refunds received; (iii) No mechanism to assure country safeguard documents redone if necessary; (iv) No Bank-wide log for procurement complaints

Highlighted

Issues relating to Results:

Documentation and potential control Deficiencies.

Highlighted Additional

Issues

55

Identified; Materiality

not yet established

Materiality should be

established during Part IB

4.7 Recommendations for Next Steps: IEG’s recommendations are focused on the issues to be dealt with in completing the remaining phases of the review, and on the broader control framework issues that may emerge going forward. Note is taken of the fact that Man-agement has identified the documentation retention and accessibility


41

as a weakness and has already launched a remedial effort. In this con-text, IEG makes six recommendations to Management, (including one also to IAD), as follows:

Confirming the Validity of the BPM Cluster: Management has argued, but has not conclusively demonstrated, that the core SIL prototype module in the cluster of BPMs can be used as a proxy for all investment type lending, because all ILs have the same controls as SILs. This proposition should be tested, and this could be done during Part IB. (para 2.18)

Reform of the OPs/BPs: IEG considers this topic an area of po-tential material weakness, whose remedy Management should treat as a priority. IEG notes that Management has a stated strategy to address the problem, both to streamline and to up-date the OPs/BPs. (para 2.37).

Completing the Remaining stages: IEG recommends that preparation for the Part II stage should begin promptly upon completion of Part I. It would seem useful to precede this work with a work plan (which could be discussed with the Board), which could benefit from consultations between Man-agement, IAD, and IEG, much as the Audit Standards were discussed under Part I. Part II should preferably be completed expeditiously, also because if it should be delayed, the con-trols parameters that were tested during Part I may have changed, and there may be difficulties integrating the two parts. (para 2.24).

Resolving Specific Issues and Potential Deficiencies (Man-agement and IAD): It is important that the several deficiencies uncovered by both Management’s assessment and IAD re-view, as listed and described in Annex C, be addressed during completion of Part IB. While some of these issues relate to lack of clarity in documentation, others to efficiency and effective-ness of controls, others are potential deficiencies in controls. It is the seriousness of the latter group—the materiality of their potential impact on risk mitigation—that must be addressed before conclusions can be drawn on the state of the overall control framework. Management and IAD should work on this together. (paras 2.41, 2.44 and 3.3, third bullet).

Managing the Risk Framework and Extending COSO: IEG be-lieves the Integrated Risk Management Framework will need to be broadened to focus also on compliance and operations reporting, and in this context, the Bank may also consider adopting the recently extended version of COSO which pro-vides for the addition of a new fourth objective (strategy—high level goals, aligning with supporting mission) and three new components to the existing five components of COSO: objective setting, event identification and risk response. (para 1.7 and Annex A paras 4-6).


42

Mainstreaming Internal Controls Reviews: IDA should begin considering the value of adopting a policy requiring: (1) ongo-ing monitoring and reporting on internal controls in the course of operations for all three COSO objectives; and (2) separate evaluations and reporting as necessary.

4.8 Postcript: A Summary of Key Steps to be Taken in Preparing for and Completing Part IB:

Content: Testing (or re-testing) key controls operation (Box 1, page 2)

Identified Deficiencies: Management, in consultation with IAD and IEG, to categorize, and establish materiality of deficiencies identified during Part IA (paras 2.41; 2.46; 3.5; 4.7; and Annex C)

Verify Validity of ILs in BPM Cluster: To show that SILs rep-resent all ILs, test a range of investment lending products (para 2.18 and Annex E)

Explicitly Address Fraud and Corruption Issues: Management to itemize controls where fraud issues are relevant, indicate mitigating controls (para 3.5 and Box 7)

Assessment of IT Controls (Optional): IAD has recommended assessment during Part IB; IEG agrees this is a scope limita-tion, but could be dealt with as an entity-level control, during Part II. (para 2.24 and para 3.5)

Advisory Panel: IEG will be convening an Advisory Panel of international experts to give an opinion on its evaluation of Part IA and Part IB. (para 1.17)

43

Annex A. The COSO1 Framework 1. Figure A1 below conceptually integrates the COSO objectives and components and the Bank’s own integrated risk management focal points. COSO defines internal control as a process, effected by an entity’s board of directors, management, and other personnel, designed to provide reasonable assurance regarding the achievement of ob-jectives in the following categories:

Reliability of financial reporting—relating to preparation of published financial statements;

Compliance with applicable laws and regulations—relating to compliance with applicable legal and regulatory framework, which in the case of IDA is taken to mean its charter and policies2; and

Effectiveness and efficiency of operations—relating to effective and efficient use of resources in meeting business objectives.

Figure A.1: Relationship of COSO Objectives, Components, and the Bank’s Risk Focal Points

2. To meet the above objectives of internal control COSO describes the following components of internal control. All five components must be present and functioning ef-fectively to conclude that internal control is effective over any of the three objectives.

COSO


Control Environment

Risk Assessment


Control Activities

Strategy Effectiveness

Financial Soundness

Stakeh

older

Sup

port

Operatio

nal Effic

iency

Reliable Financial Reporting

Compliance w

ith Internal Policies

and Procedures

Efficie

ncy a

nd E

ffecti

vene

ss of

Opera

tions

COSO


Control Environment

Risk Assessment


Control Activities

Strategy Effectiveness

Financial Soundness

Stakeh

older

Sup

port

Operatio

nal Effic

iency

Reliable Financial Reporting

Compliance w

ith Internal Policies

and Procedures

Efficie

ncy a

nd E

ffecti

vene

ss of

Opera

tions

COSO Objectives

Risk Focal Points

COSO Components

COSO Objectives

Risk Focal Points

COSO Components

Annex A The COSO Framework

44

Control Environment. The control environment sets the tone of an organization, influencing the control consciousness of its people. It is the foundation for all other components of internal control, providing discipline and structure. Control environment factors include: the integrity, ethical values, and competence of the people; management’s philosophy and operating style (tone at the top); the way management assigns authority and responsibility and organizes and develops its people; and the attention and direction provided by the board of directors.

Risk Assessment. The Bank defines risk as anything that hinders the ethical achievement of sustainable business objectives and results. This includes failure to exploit opportunities and to maintain organizational relevance. (See below.) Every organization faces a variety of risks from external and internal sources that must be assessed. A precondition to risk assessment is establishment of business objectives that are internally consistent and aligned with an organization’s strat-egy and mission. Risk assessment is the identification and analysis of those risks that potentially jeopardize the achievement of business objectives. Risk assess-ment forms a basis for determining how risks should be managed, and as the Bank operates in a complex and rapidly changing environment, it is critical that risk assessment and risk mitigation are rigorous and ongoing processes.

Control Activities. Control activities are the policies and procedures that help ensure that management directives are carried out. They help ensure that neces-sary actions are taken to address risks to achievement of the objectives. Control activities occur throughout the organization, at all levels, and in all functions. They include a range of activities such as approvals, authorizations, verifications, reconciliations, reviews of operating performance, security of assets, and segre-gation of duties. In terms of the Bank’s operations, control activities can include key processes such as supervision of projects, and regional portfolio risk reviews.

Monitoring and Learning. Internal control systems need to be monitored—a process that assesses the quality of the system’s performance over time. This is accomplished through ongoing monitoring activities and separate evaluation. Ongoing monitoring occurs in the course of business operations. It includes regular management and supervisory activities and other actions personnel take in performing their duties. The scope and frequency of separate evaluations will depend primarily on an assessment of risks and the effectiveness of ongoing monitoring procedures. Internal control deficiencies should be reported “up-stream,” with serious matters reported to top management and the board of di-rectors.

Information and Communication. Pertinent information must be identified, cap-tured, and communicated in a form and timeframe that enables people to carry out their responsibilities. Information systems produce reports containing opera-tional, financial, and compliance-related information that make it possible to run and control the business. They deal not only with internally generated data, but also with information about external events and activities and conditions neces-sary for informed decision-making and external reporting. Effective communica-tion must also occur in a broader sense, flowing down, across, and from the bot-tom upward in the organization. In a healthy control environment, communications are open and when a business objective is in jeopardy “bad news” flows rapidly so that corrective action can be taken in a timely manner. All personnel must receive a clear message from top management that control re-


45

sponsibilities must be taken seriously. They must understand their own role in the internal control system, as well as how individual activities relate to the work of others. They must have a means of communicating significant information up-stream. There also needs to be effective communication with external parties such as suppliers, regulators, shareholders, borrowers, donors, and interested parties such as Non-Governmental Organizations (NGOs).

3. The Bank faces a wide range of increasingly complex risks. Sound risk manage-ment must be comprehensive to meet the dual requirement of development effective-ness and resource mobilization. To ensure that opportunities are not missed and the full range of risks is considered, the Bank has organized its risk management around the fol-lowing four focal points.

Strategic effectiveness. Success in this area means setting and maintaining the Bank’s strategic direction on a course that leads to enhancing development effec-tiveness and reducing poverty. It involves making sure that strategic choices and follow-up decisions are not only in response to immediate challenges but also aim to maintain the Bank’s relevance and its role as a leading development insti-tution. Four main dimensions are included in strategic effectiveness: selectivity, execution of strategy, agility, and governance.

Operational efficiency. This is relevant to virtually all parts of the Bank’s internal operations. It is about “doing things right” in all areas, from country work to cen-tral services, from HR to finance, etc. Operational efficiency requires operating and managerial standards as well as implementation and control mechanisms that ensure reliable, high-quality, and cost-effective performance. Five main di-mensions are included in operational efficiency: human resources, fiduciary per-formance, safeguard quality and performance, operational performance, and in-formation technology.

Stakeholder support. The support of a wide range of stakeholders is key to the Bank’s pursuit of its mission. Building political will and constituencies entails close cooperation among all branches of government, NGOs, people affected by Bank projects, private firms, Bank staff, media, and others. The point is not to please everyone, but to ensure that the perspective of all relevant stakeholders has been duly taken into account in Bank decisions. Two main dimensions are included in stakeholder support: member governments and other stakeholders.

Financial Soundness. The Bank has to ensure that its financial policies and prac-tices (for investment, borrowing, and lending decisions) provide sustained access to low-cost assistance for its borrowing members, consistent with its mission and strategy. Three major sources of threats to financial soundness are country (sov-ereign) credit risk, market risk, and liquidity (funding) risk.

Recent Developments in COSO 4. While the Bank has taken a number of initiatives to develop an integrated risk management framework (as is referenced in several places in the text of this report), there are still some questions as to how effectively the integrated framework is operating and how it will develop going forward. IEG believes it worthwhile to draw attention to some of the changes that COSO has recently introduced, as a pointer for further exten-


46

sions of the Bank’s own efforts in this area. In 2004 COSO itself published a paper enti-tled Enterprise Risk Management—Integrated Framework, 3 which includes consideration of risk appetite, strategy, and decisions; new opportunities; and deployment of capital—elements which are directly parallel to those the Bank has already introduced into its own Integrated Risk Management Framework (IRMF). To emphasize the need to focus on the entity’s mission the new framework also adds a fourth objective to the earlier three objectives of operations, compliance and financial reporting: Strategic—high-level goals, aligning with supporting mission

5. Lastly, the new COSO framework expands its components from the existing five to eight, now encompassing the following:

Internal Environment—management sets tone for organization, defines how risk is viewed and addressed

Objective Setting—Objectives must exist to identify events, internal and external, that may affect their achievement

Event Identification—Internal and external events identified that create risks and/or opportunities

Risk Assessment—likelihood and impact of risk assessed Risk Response—management selects risk responses Controls Activities—policies and procedures established for risk response Information and Communication—information flows up, down, and across the

organization Monitoring—Ongoing monitoring and periodic, in-depth evaluations.

6. As described above, the COSO framework is a dynamic framework, which has been adapted to the Bank and IDA, and which its originators are also constantly adapt-ing to global developments. As the Bank enters into the next phase of its controls review, in which the COSO framework will be center stage, it may consider making similar ad-ditional adaptations to those recently suggested by COSO itself. These are intended to sharpen the focus on risk issues, and risk is what lies at the center of all internal control systems.

NOTES 1. COSO: Committee of Sponsoring Organizations of the Treadway Commission, 1992.

2. As described in Chapter 2 (para 2.4) there was a need to adapt the normal meaning of compli-ance under COSO to the case of IDA, since the latter is an international agency under its own le-gal charter, not subject to local laws in the way would be a domestic commercial organization.

3. COSO, September 2004

Annex B Standards for Assessing Deficiencies and Weaknesses

47

Annex B. Standards Agreed by Management, IAD and IEG to be used in Assessing Deficiencies, Significant Deficiencies and Material Weaknesses

IEG conducted considerable research into the question of what audit standards would be appropriate to govern this review of IDA controls. The issue was: Management had pro-posed that it would use virtually the same standards as those that it used for its assess-ments of internal controls over financial reporting, whereas this review was to be con-cerned with operational and compliance reporting, where the issues would be different. Following this research, and extensive discussions that were held between Management, IAD and IEG, it was agreed that a common standard would be used by all three parties, and what follows explains this process and the content of the standards.

1. The Bank is currently performing its assessment of internal controls over external financial reporting using existing auditing standards on attestation of internal controls over financial reporting as prescribed by generally accepted auditing standards. In per-forming its review of compliance with IDA’s charter and applicable internal policies and procedures, Management plans to use the same concepts as those defined in the Auditing Standard No. 2 (AS2) An Audit of Internal Control Over Financial Reporting Performed in Conjunction with An Audit of Financial Statements, issued by the U.S. Public Company Ac-counting Oversight Board (PCAOB) in response to the provisions of Section 404 of the Sarbanes-Oxley legislation as much as possible.

2. Management believes that applying the concepts that have been defined by audit standard setters for assessing internal controls over financial reporting will provide the level of comprehensiveness, rigor and consistency required in its self-assessment of in-ternal controls over compliance with IDA’s charter and applicable internal policies and procedures.

3. During our work it is anticipated that Management will discover items that rep-resent deficiencies and which may or may not require remediation. A control deficiency exists when the design or operation of a control does not allow management or employ-ees, in the normal course of performing their assigned functions, to prevent or detect noncompliance on a timely basis.

A deficiency in design exists when (a) a control necessary to meet the control ob-jective is missing, or (b) an existing control is not properly designed so that, even if the control operates as designed, the control objective is not always met.


48

A deficiency in operation exists when a properly designed control does not oper-ate as designed, or when the person performing the control does not possess the necessary authority or qualifications to perform the control effectively.

4. Control deficiencies are classified as one of the following: (i) an internal control deficiency; (ii) a significant deficiency1; or (iii) a material weakness.2 The classification of the deficiency is based upon the likelihood of occurrence/noncompliance and/or the significance of noncompliance.

5. Conclusions about what constitutes a material weakness over compliance or op-erations are judgmental, more so than in the case of material weaknesses in financial re-porting. Therefore, the definition of material weakness needs to be adapted from the context of the financial reporting definition, with its reliance on materiality in relation to the financial statements, to one using more judgment as to whether the operations and compliance objectives of internal control are met. To guide financial auditors in making these judgments, AS2 identifies examples of attributes the auditor should consider in evaluating identified internal control deficiencies to determine whether the deficiencies, individually or in combination, are significant deficiencies or material weaknesses. Management, IAD and IEG have agreed that clearly defined measures be established for judging operational materiality. These measures will be used as guides by each of the three groups in determining whether identified internal control deficiencies in compli-ance constitute significant deficiencies or material weaknesses. Identified deficiencies could be significant deficiencies or material weaknesses where the control deficiencies have attributes that could:

impair the achievement of IDA’s objectives, violate requirements of IDA’s charters or other contractual agreements, significantly weaken safeguards against waste, loss, or unauthorized use of

funds, property, or assets, involve conflicts of interest, involve systemic problems in country assistance, partnerships and project lend-

ing, or require the attention of Senior Management, the Board as well as the awareness

of external stakeholders.

6. All deficiencies identified during Management’s assessment will be placed on a summary deficiency schedule. The deficiency schedule will outline Management’s as-sessment of the deficiency (type of deficiency), any mitigating controls over the defi-ciency, the potential financial impact, if any, the impact from a non-financial perspec-tive, and management’s determination of how to address the deficiency, i.e. corrective action (remediation). A control deficiency or combination of control deficiencies that, in management’s judgment, represent significant deficiencies in the design or operation of internal control that could adversely affect the organization’s ability to meet its internal control objectives is a “Significant Deficiency.” A significant deficiency or a combination of significant deficiencies that Management determines to be significant enough to be reported outside IDA shall be considered a “Material Weakness.”

7. Management (i.e. Controllers (CTRVP) and Operations Policy and Country Ser-vices (OPCVP)) will prepare a report assessing the overall deficiencies and make a de-


49

termination on the impact the deficiencies have individually and in total on the internal controls over IDA’s compliance with its charter and applicable internal policies and pro-cedures. This report which will include: (i) Management’s assessment of IDA’s compli-ance with its charter and applicable internal policies and procedures; and (ii) a descrip-tion of any significant deficiencies or material weaknesses identified through its assessment, together with their respective remediation plan.

Annex C Illustration of Potential Internal Control Design Weaknesses

50

Annex C. Illustration of Potential Internal Control Design Weaknesses Findings from Management’s Assessment, the IAD Review, and the IEG Evaluation 1. In its report, Management highlighted five high-level findings where potential deficiencies and weaknesses had been revealed, as follows: (1) difficulties experienced by Management during its assessment in obtaining timely access to relevant documents, (2) changes introduced on the ground that are not consistent with policies and proce-dures, (3) the difficulty staff is having in identifying which policies they must comply with, (4) confirmation that many staff find the existing processes and documentation re-quirements very onerous and inefficient, and (5) a disparity between SIL and DPL op-erations in the extent to which corporate reviews are initiated.

2. In addition to the above five main findings, Management identified and pro-vided IEG a list of “potential issues” identified during its Part 1A compliance assess-ment work. IAD also identified and is tracking “deficiencies” based on its review of all modules as of September 20, 2006. IAD provided its list of deficiencies to Management and IEG. IAD stressed to Management the importance of evaluating and classifying (based on their level of significance) all of the deficiencies identified by both IAD and Management in order to substantiate the findings in later work and to draw conclusions.

3. IAD provided management and IEG with a list, taken from its Deficiency Tracker, of potential additional deficiencies in Management’s documentation of process flows for key controls and/or design gaps in key controls without adequate mitigating controls. The list identifies several which relate to process documentation, others to de-sign gaps, and a few to both process documentation and design gaps. IAD noted that, for each of the deficiencies, it had recommended that Management revise the process documentation and, where appropriate, clearly indicate how the associated risks are mitigated.

4. During its evaluation, IEG examined the narrative descriptions and flow charts of IDA business processes, including key internal controls, provided by Management. IEG representatives also attended meetings arranged by Management and observed its process of assessing the design of key internal controls. In addition, IEG reviewed the lists of potential issues and deficiencies provided by Management and IAD. IEG notes that Management is to do further work on these issues to establish whether the issues, or combination of issues, may rise to the level of a deficiency, significant deficiency, or ma-terial weakness, as defined by the agreed standards described in Annex B.


51

5. What follows are some examples of the main issues that IEG considers to merit attention:

Potential internal control issues or deficiencies identified by Management and/or IAD;

Additional issues identified by IEG as potential internal control issues; Issues identified by IEG as deficiencies in the materials submitted by Manage-

ment as part of its Part IA assessment.

A. Potential Control Issues Identified by Management and IAD IDA RESOURCE ALLOCATION MODEL (MODULE 1) 6. The IDA resource allocation model is a management tool used as part of a process for providing financing, in line with IDA’s Articles of Agreement, for purposes that are of high development priority in the less-developed areas of the world that are within the As-sociation’s membership. Management identified and assessed three business processes (of the 30 in total) and three key controls over IDA resource allocations. One of the key con-trols assessed was the need for the validation by the FRM Manager of all assumptions and conditions used in running the allocation model, such as whether proper allocations are made for large blend countries (e.g. India, Pakistan, and Indonesia); maximum allocations are not exceeded; and allocations for post-conflict countries are proper.

7. Information developed by Management indicates two potential issues exist re-lating to the above-mentioned internal control over IDA resource allocations. First, the FRM Manager’s validation of the allocations does not consider inputs to the allocation process but rather is based only on the outputs from the allocation model. As a result, Management cannot be assured from the validation process that the assumptions and other conditions or factors going into the model are valid. Second, responsibility for running the allocation model is assigned to a single individual, an FRM Management In-formation Specialist. While this may assure control over access to the model and what changes are made, a deficiency may exist relating to reliance on a single person for such an important step. This person should have a back-up person who is also knowledge-able about the model in case the primary person is unable to run the model.

8. IAD identified additional deficiencies relating to Management’s process docu-mentation and key controls for the FRM resource allocation processes. For example, Management did not identify any key controls for the IDA Post Conflict Allocation process and had not linked the process to OP/BP 2.30, Development Cooperation and Con-flict. IAD said the post conflict allocation model would operate in conjunction with the underlying principles of OP/BP 2.30 and recommended that Management include this OP/BP in the assessment.

CORE INVESTMENT LOAN PROCESS (MODULE 5) 9. This highly important business process module involves all aspects of develop-ing and executing the project cycle for a Specific Investment Loan (SIL) and includes nine key controls.


52

10. Based on the information it gathered, Management noted that in certain instances reviewing officials in the loan department, procurement, financial management, or legal may approve loan documents but include, with their approvals, “subject to” comments. The comments require certain actions to be taken before the documents are processed to their final stages. However, procedures did not provide for the reviewing officials to fol-low-up on their “subject to” comments to ensure that relevant documents included changes suggested or that the comments were in some way appropriately addressed. In addition, procedures allowed certain project changes to be initiated between the TTL and the borrower that could have legal and/or disbursement implications but the changes did receive appropriate legal and/or loan department review and authorizations.

PROCUREMENT MANAGEMENT (MODULE 14) 11. The procurement regime for a SIL is covered in this process, including nine con-trols to ensure disbursement of IDA funds to suppliers of goods and services in accor-dance with Bank procurement guidelines, to achieve equity and efficiency in procure-ment practice, and to take account of local capabilities in executing contractual awards and monitoring in accordance with Bank guidelines.

12. Based on information gathered, Management identified several potential control issues including these five: (1) there could be a potential conflict of interest when a Task Team Leader (TTL) is also a procurement accredited staff (PAS) and thus performs many of the procurement functions on his or her own without an appropriate segrega-tion of duties; (2) when the TTL and procurement staff are nationals of the project coun-try there is a potential for collusion and conflict of interest; (3) prior review may not be taking place as designed by IDA policy if implementing agencies structure contract packages in such a way as to avoid it (e.g., splitting large contracts into multiple smaller ones solely to avoid prior review requirements, or amending a contract after award to an amount that would have required prior review); (4) staff completing the form 384, which authorizes disbursement on contracts that required prior review may not be appropri-ately knowledgeable or accredited to do this (sometimes TTLs request other staff to do it, and some staff who had the authorization in the past and no longer have it may not have been removed from the list of authorized users), and appropriate staff do not re-view the forms; and (5) contracts are selected for post review subjectively by TTLs or procurement staff, which leaves the process open to potential manipulation.

13. IAD identified numerous documentation and/or potential control design deficien-cies relating to procurement management. As one example, Management did not identify as a key control the mandatory step prescribed in BP11.00, Procurement, that the PS or PAS clear a General Procurement Notice issued by the borrower. IAD said this step is critical for ensuring timely and transparent notification of bidding opportunities. IAD recom-mended that Management designate the required clearance step as a key control.

LOAN AMENDMENTS (MODULE 21) 14. The amendment module describes the review of the Loan Department staff when TTLs submit proposed amendments to IDA credit agreements that affect the disburse-ments schedule. The objective of the review is to ensure that amendment documentation is consistent with IDA policies. Management identified two key controls in the amend-ment process.


53

15. The Loan Department has no control, such as a log of amendments requested, to ensure that all amendments requested are acted on. Thus, some amendments that are requested to the Loan Department may not be processed and approved, and the original disbursement schedule may not meet the needs of the borrower.

QUALITY ASSURANCE GROUP (MODULE 27) 16. Activities of the Quality Assurance Group could be viewed as an entity-wide monitoring control within the Bank. Its importance as a control rests largely on the im-pact QAG results can have both on the management of individual IDA projects and on operations across the Bank. QAG serves as a corollary check on the quality of the work performed by line employees at the entry (QEA) and supervision (QSA) phases of the lending cycle. The results of QAG work are communicated both during the assessment process and also at the end in individual and “synthesis” QAG reports.

17. Management identified a potential issue in that it found no control to ensure QAG recommendations are implemented.1 The descriptions in Management’s documen-tation do not sufficiently address how recommendations and other QAG results are to be used to effect improvements in IDA operations. Specifically, the synthesis phase of the narrative and mapping do not adequately explain how recommendations are to be acted on, tracked, and disposed of.

18. IAD identified a potential control deficiency relating to the selection of QAG panel members. Management had identified the risk of individuals serving on the pan-els who may not be independent and objective vis-à-vis the project they are assessing. Management had not identified a key control to address this risk. IAD recommended that Management identify and assess a key control to ensure QAG panelists are inde-pendent and objective relative to the project being assessed.

B. Potential Control Issues Identified by IEG 19. IEG concurs with the issues that were identified by Management as potential control deficiencies and has discussed these with Management. IEG agrees that they should be examined further and remedial action taken where needed. In addition, IEG highlights the following issues:

PROCUREMENT COMPLAINTS (MODULE 15) 20. The procurement complaints module is the process performed by procurement staff and the TTL to manage complaints received from within or outside IDA regarding procurement. The objective of the two controls in the module is to ensure that procure-ment complaints regarding procurement on SIL projects are addressed and resolved in accordance with IDA policy and that decisions are disseminated to appropriate internal and external parties.

21. Information gathered by Management revealed two potential internal control weaknesses. First, there is no control to ensure that all complaints are entered into the complaints database. Having all complaints in the database is the first step in ensuring that all complaints are handled appropriately, such as by being referred to the Department of Institutional Integrity and considered for potential non-compliance with IDA procure-


54

ment policies. Second, there is no control to ensure that reports from the complaints data-base are followed up on to ensure all complaints are handled appropriately, because there are no regular reports produced from the complaints data base. Because of these weak-nesses, the monitoring control that complaints themselves provide to IDA may not func-tion as effectively as it should and problems in procurements may go unaddressed.

DISBURSEMENTS AND LOAN AMENDMENTS (MODULES 19, 21) 22. These two modules describe the reviews of the Loan Department staff when (1) borrowers submit applications for disbursements on their IDA credits or requests for special commitments or (2) TTLs submit proposed amendments to IDA credit agree-ments that affect the disbursements schedule. The objectives of the reviews are to ensure that (1) disbursements are made based upon complete, accurate documentation that is properly approved in accordance with the credit’s financial agreement and (2) amend-ment documentation is consistent with IDA policies. Management identified five key controls in the disbursement process and two in the amendment process.

23. IEG found two potential deficiencies in control design in these two modules:

The TTL is allowed much flexibility in the Operational Policies that govern the staff’s IDA activities. The TTL may exceed a category amount with the under-standing that he or she will input an amendment later. However, IEG could find no control in either module to ensure that the later amendment takes place. Thus a change may be made to an IDA credit disbursement schedule without the ap-propriate approvals and documentation change.

IEG did not identify any standard mechanism for a TTL to let Loan Department staff know if he or she has concerns about a project and wants to put a hold on a disbursement. Thus, a disbursement may be made on a credit when the TTL has some reason to believe it should not be made.

REFUNDS AND LOAN ACCOUNT CLOSING (MODULES 22, 25, 26) 24. The refund and loan closing modules include activities of the Loan Department staff to process the closing of IDA credits after the final disbursement or to process re-funds received by the Bank for funds previously disbursed. The objectives of these ac-tivities are to ensure that IDA issues the appropriate notifications to the borrower, that any credit balances are cancelled in the loan system and the credit is officially closed, and that any special accounts with a balance remaining are refunded to IDA. Manage-ment identified one key control in the refund process and two key controls in the stan-dard loan closing process and two in the special closing process.

25. IEG learned that the Loan Department does not have any mechanism to ensure that balances remaining in special accounts that should be refunded to IDA are eventu-ally received by the Bank. Loan Department staff notify the borrower with a balance in a special account of payment instructions, but there is no database monitored to ensure all refunds are received. Instead, individual Finance Officers can keep notes of the refunds expected in their portfolios. Management did not recognize this as a deficiency.


55

SAFEGUARDS (MODULE 28) 26. The safeguards process includes the activities performed by the IDA safeguards personnel for SILs from project identification to completion. The objective of the process is for safeguards personnel to (1) ensure that safeguard aspects of a project are assessed appropriately given the project circumstances and design; (2) for all safeguards policies that are assessed as being triggered within a project, ensure project design and imple-mentation arrangements are adequate to comply with IDA policy and procedures; and (3) monitor safeguards aspects of projects to ensure the arrangements are carried out ac-cording to IDA policy and procedures and adjusted as necessary.

27. IEG recognized that there is a gap in the process documented by Management in that if certain safeguards documents in the preparation phase are inadequate (such as the environmental assessment, done by the country), there is no mechanism in the proc-ess for it to be sent back to be redone. This is necessary to mitigate the risk Management has identified that client commitment and capacity to implement safeguards may not be sufficient to ensure that the safeguards are implemented. This in turn could result in re-putational risk to IDA.

C. IEG Identified Deficiencies in Management’s Documentation COUNTRY ASSISTANCE STRATEGY (MODULE 4) 28. Management identified and assessed a business process module and three key controls related to the preparation of the country assistance strategy (CAS), the central tool with which Management and the Board review and guide IDA’s support for the country’s development programs. The objective of the process is to ensure alignment of the CAS with associated lending volume, most recently approved Performance Based Allocation, creditworthiness, potential legal issues, and overall CAS quality.

29. Management’s process maps and descriptive materials omitted from the CAS Preparation phase what IEG considers to be key steps and related controls relating to (1) portfolio/pipeline review, (2) Analytical and Advisory Activities including Country Economic Memorandum actions, and (3) the Poverty Reduction Strategy Papers.

30. IEG could also not find in the CAS BPM any reference which explicitly identified the inputs of the Sector Networks into developing the CAS.

CORE DEVELOPMENT POLICY LOAN PROCESS (MODULE 7) 31. This business process module involves all aspects of the cycle for executing a de-velopment policy loan (DPL) from identification to completion and includes seven key controls. The objective of the process is to ensure DPL operations are identified, exe-cuted, and completed according to IDA policy and procedures with inputs as required and necessary from internal and external parties.

32. Management’s statement of risks does not address the potential impact on IDA’s reputation and the country if the DPL objectives are not accomplished. The limitations of country capacity, structure, etc. that the DPL is intended to fill will continue to exist if DPL activities are not completed or for any reason, do not accomplish the objectives for


56

which the DPL was approved. IEG considers this a major risk that is not identified in the Management assessment.

LOAN MANAGEMENT (MODULES 17& 18) 33. The business process modules involving all activities performed by the Loan De-partment in the preparation of individual SILs and DPLs from project identification through loan effectiveness (for SILs) or supervision (for DPLs) include five key controls in the SIL process and six in the DPL process. The objective of the Loan Department ac-tivities is to ensure that each project includes in the financing agreement and the pro-gram document the appropriate overall financial structure of the loan including dis-bursement and repayment arrangements, as required by IDA’s internal policies.

34. However, Management did not consider the risks in the area of repayment and amortization in these modules, which IEG considers to be a major risk area in these ac-tivities. Even though repayment terms are standardized in IDA, there should be a place to mention repayment flows as an issue in the loan process flow charts, since there is al-ways some risk involved.

QAG PROCESSES (MODULES 5, 7, 10, 11, 12, 13, 14, 17, 18, 28) 35. The QAG processes are those that monitor, in real time, assessments of projects which are still active, all key aspects of project quality, including quality at entry (QEA) and during supervision (QSA), in all aspects of the Banks’ tracking and supervision, in-cluding all fiduciary, contractual and safeguards aspects.

36. Management correctly identifies and maps these QAG processes in the QAG module (Module 27), but it does not make explicit the QAG interventions in the other in-dividual line operations modules. This does not affect the assessment of the design effec-tiveness of the controls, but it is a deficiency in the completeness and accuracy of the proc-ess flow charts in the modules mentioned, and by omitting these references Management misses an opportunity to emphasize that the QAG assessments cover all aspects of project design and implementation, including the fiduciary, contractual and safeguards.

57

Annex D. A Typical BPM: Descriptive Material Extracted from Management BPM Materials

Figure D.1: Sample Process Map for Module 8—Corporate Review of a SIL Operation

Annex D Typical BPM: Descriptive Material

58

IDA 14 INTERNAL CONTROL REVIEW MODULE #8

CORPORATE REVIEW PROCESS OVERVIEW

Description of Process

These are the activities associated with conducting a corporate review, with the op-erations committee (OC) or the regional operations committee (ROC), during the identification or preparation phases of an operation or country assistance strategy (CAS).

Objective of the Process

For DPL operations and CAS(s) the corporate review ensures that documents are reviewed by senior management at the corporate level during identification and / or preparation. For SIL projects, the corporate review ensures that higher risk and exceptional projects are reviewed by senior management, though this is not manda-tory and can also happen after preparation, if appropriate.

Risks Impacting the Process

General Risk as Defined in the IRMF: • Choices within countries (2) • Quality assurance (5) • Policy / guideline alignment (7) • Results-oriented culture (12) • Timely evaluation and disclosure (13) • Reliable procurement / disbursements (18) • Safeguards compliance (22) • Management of projects with complex safeguards issues (23) • Business process management (24)

Specific Control Risks: • IDA projects are not focusing enough resources on priority activities within a

country. High risk and high profile projects within a country may not receive the appropriate level of management review to ensure management buy-in for the re-sources required to adequately fund or scale up these projects / operations

• Not enough of a visible layer of review for quality of operations and their fiduciary arrangements. Innovative or new designs may not be reviewed for quality and fiduciary soundness

• Those operations with the highest potential risk and impact both within the institu-tion and within the borrower country may not be given the highest possible visibil-ity and review at the earliest possible stage, relative to IDA’s other ongoing projects and operations. It is of the most importance for these operations that risks and problems be identified early and have had the most senior management visibility


59

• For high-risk safeguard projects, there may be no opportunity for corporate buy-in, and projects may be rejected because of their high reputation risks

• The business process is made to be too cumbersome for the relative risks and requirements associated with a project / operation. Not every project / operation requires an OC-level review, and if all were tapped to receive one this would result in an unnecessary bottleneck in the process

• For All: Lack of coherence between proposed operations and the CAS and sectoral strategies

• For CAS products: The mitigation of all aspects of risk associated with choices within countries may not be carried out –IDA’s efforts in individual countries may not be sufficiently concentrated on priority activities where we can have the great-est impact—including through scaling up efforts. Resources may be spread across too many activities instead of adequately funding critical work. IDA’s mix of in-struments within countries may not be appropriate.

• For CAS products: Activities programmed by IDA are not harmonized with other development partners

• For DPL and IL: No consistent application of IDA policy and adherence to IDA guidelines, nor the opportunity for senior management to help resolve policy is-sues

• For DPL and IL: Lack of linkage between proposed operations and targeted results and / or objectives

• For IL: No review of project design parameters, implementation plans, and integration of capacity building in an operation

Mitigating Key Controls Identified

In order to ensure the above risks are mitigated, the procedure for deciding on if / when and how to hold a corporate review was reviewed by subject matter experts who prepared the attached process flow diagram and identified the following key control steps in the process: • Decision to hold ROC or OC review meeting • ROC Review / OC Review


60

Key Control Detail Sheet

Corporate Review Corporate

Review—Control 1

Project Stage: Identification or Preparation

Key Control Step: ROC or OC?

A. Description:

For Projects/Operations:

Corporate Review level is decided by the Region (Task Team Leader (TTL), Country Director (CD), Regional Vice President (RVP)) in consultation with Operational Policy and Country Services (OPCS). The Operations Committee (OC) Secretary and the Managing Director (MD) are notified of the decision.

The decision is based on the criteria as outlined below, from the online guidelines.

From Guidelines:

• OC review is recommended for operations that:

o pose high risk for the institution; o have a large size or represent a substantial departure from the CAS; o are subject to OC review under existing policy and guidance; and o facilitate institution-wide learning or adequate sampling to review for quality.

• (i) Risk: An OC review may be appropriate when there are high risks, includ-ing:

o Safeguard Risks and Corporate / Reputational Risks. Operations with critical safeguard and / or reputational risks, as identified through the existing process (whereby Regions flag operations with high safeguard and reputational risks to ESSD/QACU and Senior Management).

o Operational / Development Effectiveness Risks. Operations with high development effectiveness risks as identified through the exist-ing process (whereby Regions flag operations with high operational or development effectiveness risks to Senior Management).


61

o Financial risks. Operations that entail financial risk or systemic con-siderations for IDA’s finances (credit loss provisions, capital needs, available net income), as identified by Credit Risk (SFRCR).

o Anti-Money Laundering (AML) / Combating the Financing of Ter-rorism (CFT) Related Risks. Operations that are exposed to high risks of money laun-dering and terrorism financing.

• (ii) Size and Departure from the Country Assistance Strategy (CAS). An OC review of an operation may be appropriate when the operation is large or when the operation diverges significantly from the program laid out in the CAS:

o Size of Operation. Given the potential financial implications for the institu-tion of large operations, the OC should generally review all IDA opera-tions of at least $200 million.

o Departure from CAS Lending Program. While most CAS implementation related issues should be handled at the ROC level, there may be substan-tial (operation-specific) departures from the CAS (in terms of financing size, instrument, risk or content) that would indicate an OC review. The OC could review certain proposed operations that involve a substantial increase or acceleration in total IDA lending relative to the CAS.

• (iii) OC review required by existing policy and guidance. These include special DPLs, and DPLs with a deferred drawdown options (DDO), crisis and post-conflict situations (and guarantees). In addition, the OC reviews operations involving exceptions from op-erational policy.

• (iv) Institution-wide learning and sampling to test quality. The OC reviews credits with new or potentially contentious approaches that are likely to create precedents for other operations in other Regions. In these cases, the OC can bring institution-wide in-novation and learning to bear on initial operations. OC review may also be appropriate to ensure that a minimum number of operations in a Region are tested for quality.

For CAS Products:

• For CAS products, an OC is recommended for the same criteria stated above for operations pertaining to items (i), pose a high risk to the institution, and if the proposed CAS lending envelope is large.

B. Control Details:

Objective: Validity / Existence Type: Manual,


62

Preventive Frequency: Infrequent Transaction

C. Risks Mitigated / Policies Addressed:

IRMF Risk(s): 1. Choices within Countries (IRMF: 2) 2. Quality assurance (IRMF: 5) 3. Timely evaluation and disclosure (IRMF: 13)

4. Reliable procurement/disbursements (IRMF: 18) 5. Management of projects with complex safeguard issues (IRMF: 23) 6. Business process management (IRMF: 24)

Specific Risk(s): 1. IDA projects are not focusing enough resources on priority activities within a coun-

try. 2. High risk and high profile projects within a country may not receive the appropriate

level of management review to ensure management buy-in for the resources required to adequately fund or scale up these projects/operations

3. Not enough of a visible layer of review for quality of operations and their fiduciary arrangements. Innovative or new designs may not be reviewed for quality and fidu-ciary soundness

4. Those operations with the highest potential risk and impact both within the institu-tion and within the borrower country may not be given the highest possible visibility and review at the earliest possible stage, relative to IDA’s other ongoing projects and operations. It is of the most importance for these operations that risks and problems be identified early and have had the most senior management visibility

5. For high-risk safeguard projects, there may be no opportunity for corporate buy-in, and projects may be rejected because of their high reputation risks

6. The business process is made to be too cumbersome for the relative risks and re-quirements associated with a project/operation. Not every project/operation requires an OC-level review, and if all were tapped to receive one this would result in an un-necessary bottleneck in the process

Policies: 1. BP 2.11—Country Assistance Strategies 2. OP/BP 10.00—Investment Lending: Identification to Board Approval 3. OP/BP 8.60—Development Policy Lending

D. Owner: Regional Vice President (RVP)

E. Other Parties:

Country Director (CD), Task Team Leader (TTL), Operational Policy and Coun-try Services (OPCS), Managing Director (MD), Credit Risk (SFRCR)

F. Verifiable Evidence: • None required


63

o Minutes of any meetings that may have been held with the OC/ROC Sec-retary are available for any such meetings that may have taken place

G. COSO Framework:

This control step contributes directly to the fulfillment of the risk assessment and con-trol activities elements of the COSO framework.


64

NOTES 1. AS2 defines a significant deficiency as a control deficiency, or a combination of control defi-ciencies, that adversely affects the company’s ability to initiate, authorize, record, process, or re-port external financial data reliably in accordance with generally accepted accounting principles such that there is more than remote likelihood that a misstatement of the company’s annual or interim financial statements that is more than inconsequential will not be prevented or detected.

2. AS2 defines a material weakness as a significant deficiency, or combination of significant defi-ciencies, that results in more than a remote likelihood that a material misstatement of the annual or interim financial statements will not be prevented or detected.

1. By contrast, in its evaluations, IEG’s main recommendations are entered into the Management Action record (MAR) which is updated annually and reported to CODE.

65

Annex E. Does the Cluster of BPMs Represent the Universe of IDA Controls? 1. Management’s claim that the 30 BPMs it identified “captured a representative picture of the control environment over IDA operations”1 was independently tested by IEG. The first step was to propose criteria (shown in Box E. 1 below) against which to measure this claim. What follows is the ensuing analysis and conclusions IEG arrived at.

Box E.1. Criteria by which to Evaluate if 30 BPMs Adequately Represent the Universe of IDA Controls PREMISE: Business processes have the purpose of achieving business objectives; the key controls embodied in the processes have the purpose of addressing the risks which arise in the pursuit of those business objectives. How to judge the adequacy of the cluster?

• Does the cluster provide a measurable basis to assess COSO compliance with internal policies and procedures? Does it adequately link, in aggregate, to the IDA risk focal points under COSO?

• Do the business processes cover all key business objectives? • Does the cluster comprise a high share of the IDA operating budget? • Does it cover all major IDA product lines? • Where modules have been excluded, does this create significant gaps in measuring compliance?

2. Compliance with COSO and Internal Policies and Procedures: In the approach which Management has taken, the cluster has not been built as an expression of entity level controls cascaded down to the operating level; rather, it has been built up from the operating level, as a means of providing the basis for testing the design and operation of mainly fiduciary controls that govern IDA lending. Management has linked each mod-ule and its controls to published policies and procedures, but some 50% of the total number of OP/Bps are not linked. However, in the cases where no linkage has been made, Management has given satisfactory explanations (most often because the OP/BPs refer to sub-processes of the main processes covered). To this extent, the cluster broadly conforms to policies and procedures. (para 2.17)

3. Management has also been clear that its focus on COSO during this Part IA stage of the review has been only partial. IEG has checked this against the links shown in the cluster, module by module. The table below shows that, taken individually, the modules have links overwhelmingly to the Control Activities of COSO, in a minor way also to the Risk Assessment, Monitoring and Information and Communications components, but with no link at all to the Control Environment . These results are not surprising, since Management has stated that the links to COSO in this Part I phase are only partial. It does show the need, however, for later stages of the review to build links to the other COSO components.

Annex E Does the Cluster of BPMs Represent the Universe of IDA Controls?

66

Table E.1. Stated Links between BPMs and COSO Components Units Denote One Stated Link per Key Control

(Some controls link to more than one COSO component)

Control environment

Risk assessment

Control activities Monitoring Information and

communications 14 109 25 11

Source: Management Business Process Modules, Control Detail Sheet 4. The BPMs and Business Objectives: Management’s building block approach, creating a hierarchy from Articles, to policies and procedures, to identifying the key business processes, supported by “umbrella” policy statements, is logical, transparent and convincing. The 30 business processes which were identified between them embody four main business processes: IDA allocation; the CAS process cycle; the two main lend-ing type (SIL and DPL); and the supporting fiduciary, contractual and safeguards proc-esses. However, there is no direct capture of non-lending processes and objectives. While some of these could be subsumed in the CAS process, in fact the treatment of CAS does not make this explicit. The lending product ands heavy fiduciary emphasis in the cluster is evident from Figure E.1 below. It shows how the majority of modules and key control points are located in the fiduciary and contractual support modules (21 out of 30 modules, and 82 out of 114 controls).

Figure E.1. Distribution of BPMs and Controls Across Business Functions

0 5 10 15 20 25 30 35

QAG Processes

Financial Management

Safeguards

Legal

Procurement

Loan Administration

Lending Products

Internal Programming

No. of BPMs & Controls BPMs Controls

Fidu

ciar

y &

Con

trac

tual

Lend

-in

gQ

ualit

yA

ssur

ance

Total Controls per G

roup

19

82

6

Allo

catio

n &

Pr

ogra

mm

ing

7

5. The Share of IDA’s Operating Budget and Product Lines: If the cluster of BPMs exactly matched the totality of IDA controls, the service costs of the business processes


67

captured in the 30 BPMs would account for the total IDA operating budget. In fact, it appears to account for significantly less than the total. From the data in Table E.2 below a broad measure can be made as follows: Lending and supervision (which are the major process in the cluster) account for 58% of the IDA country service spending over the past three years. To this can be added some unknowable portion of the 20% for “Other” ex-penditure—to cover the overhead costs of the IDA allocation and CAS products proc-esses which are represented in the cluster. If the whole 20% were added, the cluster would represent 78% of the total IDA operating budget, which is a substantial portion. However, the budget share of AAA (22%) is almost the same share as that for lending preparation (24%), so on these grounds alone, it cannot be claimed that the cluster cap-tures “a representative picture” of IDA controls. There are also other reasons (argued in the main text—see para 2.20) why IEG believes it unjustified to have excluded AAA and other non-lending products.

Table E.2: Country Service Costs for IDA Countries 2003-2005

(US$M) 2003 2004 2005 3yr Av.% Service Costs for IDA (only) countries 180.7 213.2 219.2 100 Of which: Project Supervision 62.7 67.0 77.1 34 Lending 49.7 65.5 59.0 24 AAA 38.2 48.5 48.7 22 Other 30.1 32.2 34.4 20 Source: Business Warehouse

6. Do SILs and DPLs Adequately Represent Lending Operations? The BPMs that Management has used to represent IDA lending are “core” SILs and DPLs. These are but two of a wider range of lending products, so there is a question whether all lending products are well represented by these two prototypes. IEG reviewed the data for the IDA lending program in recent years. It showed that all adjustment lending is captured under DPLs. The investment lending is shown under SILs and other types of investment lending. The share was 64% for SILs, with other investment loans comprising 36%. Man-agement argues that all lending products—whether IL or DPL—share almost identical processing and controls. In IEG’s opinion this needs to be tested, because while it may be broadly true, there are also variations in lending product type which may be proc-essed somewhat differently through the controls system. This could be done during Part IB. The test samples for lending products could include a range of product types, not just core SILs and DPLs.

Table E.3. Share of IDA Investment and Policy Loans Investment Lending(2005 $m) Core SIL Other Investment Lending

5626 3312 2314

3-Yr Av.% 64 36

Adjustment Lending (DPL) 2161 100 TOTAL LENDING 7787 Source: Business Warehouse

7. Excluded Processes: Management gives the following reasons for excluding cer-tain processes or product lines from the assessment during this phase of the study:2


68

a. that they do not contribute directly to IDA lending operations; or

b. they are not relevant to this phase of the study, because they relate more to effi-ciency and effectiveness issues than to compliance.

Box E.2. Business Process Modules Excluded from Compliance Assessment Exclusion By Management’s Reso-lution That the Process Does Not Have Critical Bearing on Current

Assessment Objective

Exclusion Based on Determination of No Input to IDA Operations

• Country Policy and Institutional Assessment (CPIA)

• Procurement DPL

• Post-Conflict Performance Indi-cators (PCPI)

• IEG Process

• Project Preparation Facility (PPF) • IAD Process

• Loan Management –PPF Refi-nancing

• AAA Products

• Annual Report on Portfolio Per-formance (ARPP)

• Inspection Panel Source: Management Methodology Note

8. A summary of the excluded processes, grouped according to the major reasons for their exclusion is given in Box E.2 above. Many of these processes relate to or form part of the Bank’s overall monitoring instruments (ARPP; Inspection Panel; IEG proc-esses; IAD processes), which are more relevant to the entity-level phase of the study, and concern more the efficiency and effectiveness objective than compliance. In this sense, Management’s justification for their exclusion has been clearly stated. Equally, the proc-esses dealing with PPF and procurement DPL are of generally secondary significance, and their exclusion would not create any significant gaps in coverage. In evaluating these exclusions, IEG makes the following observations:

i. Had all processes been subjected to a top-down, entity-level review process to start with, it would have been easier to make a consistent and complete justifica-tion for excluding certain processes less relevant to business process controls; as it is QAG processes are also an entity-level control, but these have been included, rather than excluded in this round.

ii. Knowledge products (specifically AAA) should clearly have been included in this compliance part of the study.

iii. Regarding CPIA and PCPI, both are sub-processes of and provide inputs to IDA allocation, and should have been included.

iv. While the Inspection Panel is a key entity-level unit, its activities always concern issues relating to complaints regarding specific non-compliance with (Bank and) IDA policies, so there would seem to be a default case for its having been in-cluded rather than excluded from the compliance part of the study.


69

9. Scorecard and Conclusions: How do these various issues tally, and what conclu-sion do they suggest in answering the main questions: does the cluster well represent the universe of controls governing IDA operations? Based on this analysis, as a represen-tation of IDA allocation and lending processes—which are the bulk of IDA operations—Management’s BPM cluster scores well on most counts and is a credible representation of the full universe of IDA lending. However, in excluding non-lending, Management has taken out of play an important set of product lines that not only rank almost equally with lending preparation costs in the operational budget, but rank alongside lending as an important part of the IDA assistance portfolio.

NOTES 1. Management Report paragraph 25 page 10.

2. As part of the working documents Management produced for its assessment, one was under the title “Methodology Note,” which outlined the reasons for these exclusions.

70

Annex F. Method and Results in Applying the Business Process Template 1. Introduction to the Business Process Template: IEG created the Business Process Template especially for this review. Its main purpose was to have an evaluation tool which would contain a standardized set of questions which could be applied uniformly to each evaluated business process module, to critique its content in a systematic way, and to provide ratings as to the quality exhibited.

2. Rating System: The Template embodies a four-part rating system which reflect quality ratings from Highly Satisfactory (1), Satisfactory (2), Satisfactory with Qualifica-tion (3) to Less than Satisfactory (4) measured as degrees of certainty that Management has achieved in its method and construct of the module, in addressing the following un-derlying question:

“What degree of certainty does Management’s Assessment (and IAD’s Review) provide that the business processes are well designed and mapped and that their associated control proc-esses are effectively designed?”

3. The Template was created for use in the IEG evaluation of all stages of the con-trols review, and it has sections dealing with compliance, operational and entity-level issues. For the present Part IA stage, only a portion of the Template was used, covering three sets of issues: ranking the BPMs according to their strategic importance and mag-nitude of risk; evaluating the method of building and mapping the BPMs; and evaluat-ing Management’s assessment of the effectiveness of key control design. What follows is a brief summary of the form of questions that the Template contained in each case.

4. Ranking for Strategic Importance and Risk: The rankings were from 1 (Highly relevant, critical, heavy weight in management, high risk); 2 (Relevant, medium weight); 3 (Relevant but not critical, low weight, low risk). The questions in the Template on this topic were as follows:

Its centrality to the overall IDA-Client relationship? Its weight as a control instrument within the Bank? The Risks that would pertain were the process to be lax? Its frequency of Occurrence? Potential magnitude in Dollars?

5. Evaluating the Mapping of the BPMs: IEG viewed the task of evaluating Man-agement’s mapping of the business process modules as having three elements, and the Template that was used as the principal evaluation tool contained questions which ex-plored Management’s methods and criteria in mapping the modules, as follows:

Annex F Method and Results in Applying the Business Process Template

71

The method and criteria used in identifying each module: Was the business objective clearly described? Were specific criteria given for choosing the module? Did the process derive from a specific published policy or procedure? Did Management con-sult the most knowledgeable people in the Bank in constructing the process map?

The Accuracy and Completeness of the Process map: Was the process clearly ti-tled, in a way which made it clear what business process was being tracked? Were the risks pertaining to the business process clearly stated? Were the units in the Bank which are key to the process clearly identified? Was the process ade-quately linked to parallel or related sub-processes?

Identification of the Key Controls: Were the key controls clearly defined? Is it evident why the key control is relevant to the process? Were links made between the COSO objectives and individual controls?

6. Evaluating Design Effectiveness of Key Controls: Most of the business process modules had multiple key controls, ranging from two or three to nine or eleven in some cases. Since the purpose of the controls is to address risks that are confronted in the pur-suit of the business objective that the process is aimed at, assessing the effectiveness of control design—and the corresponding structure of the Business Process Template—is basically an issue involving two aspects:

Identifying Process Risks: Were the specific risks identified that the control was intended to mitigate? Were the risks categorized as to type and priority or likeli-hood? Were the risks derived from published policies and procedures? Were the most authoritative sources consulted?

Matching Risks with Process Design: In matching risk and controls, were there adequate checks and balances, and were there any gaps? Were specialist staff in-volved? Does the control oblige involvement of specific units and management authority? Is the design of the control widely known? Does the control relate also to risks external to the process and the Bank? Did Management identify specific weaknesses or deficiencies in the design of the controls?

7. Summary of results: A full summary of the ratings data is provided in the Statis-tical Appendix at Annex G. The table below gives a summary of the results, showing the average ratings achieved for each segment, and a set of selected highlights showing higher and lower quality aspects.

Table F.1. Management’s Method and Approach to BPM Mapping and Control Design Selected Average IEG Ratings

ALL MODULES Distribution By Rating 1 2 3 4

Distribution by Number 15 450 184 51 Distribution by Percentage (%) 2 64 26 7

Higher Quality Lower Quality Mapping the Business Process: (Average Overall Rating 2.38) Selected Higher Quality Ratings Clarity in Method and Criteria Ownership of Process Relevance of BPM Controls Design Process Widely Known Coverage of External Risks

Av. Rat-ing - 2.00 2.00 2.04 2.08 2.08

Assessment of Control Design: Average Overall Rating (2.62) Selected Lower Quality Ratings Categorizing Risk by Type and Likelihood Matching Design with Risks Risks to BPM clearly stated?

Av. Rat-ing 4.00 2.81 2.73

Annex F Method and Results in Applying the Business Process Template

72

8. This Template analysis has shown that Management’s methods in both develop-ing the process flow charts and associated materials, and assessing the effectiveness of control design, has been conducted to a high standard, even though in some cases im-provements could be suggested. As the table shows, fully 66% of all ratings were Satis-factory or better, and some 93% were Satisfactory with Qualification or better. Most of the reasons which underlay the areas where qualifications were made referred to descrip-tions of processes, and categorization and prioritization of risk. Other imperfections in the precision of the documentation included: the CAS module omitting portfo-lio/pipeline review, AAA, and the inputs of the Sector Networks; the DPL module not addressing potential impact on IDA’s reputation and the country if the DPL objectives are not accomplished; Loan management modules not considering the area of repay-ment and amortization; and the processes subject to QAG assessments not being shown on the main flow charts.

9. Since this was the first time the Template was used as an evaluation tool its use-fulness was also being tested in this process. In the view of the panelists involved in the ratings, the Template was responsive and gave credible, robust results.

73

Annex G. Statistical Appendix TABLE G.1: IDA OPERATIONS DATA REFLECTING MAGNITUDE OF SELECTED BUSINESS PROCESS MODULES, 2003-2005 TABLE G.2: LINKS IDENTIFIED BY MANAGEMENT BETWEEN CONTROLS AND THE FIVE COSO COMPONENTS, SHOWN BY BUSINESS FUNCTION TABLE G.3: CATEGORY AND TYPE OF KEY CONTROLS, BY AUDIT OBJECTIVES, SYSTEM AND FREQUENCY TABLE G.4: SUMMARY OF QUALITY RATINGS OF MANAGEMENT’S ASSESSMENT OF THE DESIGN EFFECTIVENESS OF KEY CONTROLS TABLE G.5: TABLE G.5: SUMMARY OF QUALITY RATINGS FOR MANAGEMENT’S MAPPING AND ASSESSMENT OF DESIGN EFFECTIVENESS OF KEY CONTROLS TABLE G.6: DISTRIBUTION OF QUALITY RATINGS ACROSS MAPPING AND CONTROL DESIGN DIMENSIONS TABLE G.7: DISTRIBUTION OF BPMS BY STRATEGIC RELEVANCE AND RISK RATINGS TABLE G.8: LISTING OF OPS AND BPS LINKED BY MANAGEMENT TO THE BPMS

Annex G Statistical Appendix

74

Table G.1: IDA Operations Data Reflecting Magnitude of Selected Business Process Modules Number of Cases US$(m) 2003 2004 2005 2003 2004 2005 Total Number of Approved IDA Projects1 141 158 158 7282.5 9034.6 8559.0 Of which: Investment Loans 117 135 126 5451.3 7336.6 6258.0 Adjustment / Dvlp. Policy Loans 24 23 32 1831.2 1698.0 2301.0 Of which: SILs 66 88 77 3342.2 4992.5 3312.1 DPLs 0 0 9 0.0 0.0 462.0 …………Active Loans 753 764 765 34,722.8 37,045.8 36,228.3 Service Costs for IDA (only) countries2 180.7 213.2 219.2 Of which: Project Supervision 62.7 67.0 77.1 Lending 49.7 65.5 59.0 ESW 38.2 48.5 48.7 IDA Allocations3 Subject to Post-Conflict Allocation (projects) 19 19 16 1235.2 1393.1 662.7 Operations Programming4 Number of CAS (full / TSS) 24 /

4 25 /

6 25 /

3

Number of PRSPs (full) 12 12 8 Cumulative Total (Full) 49 (Interim) 10 IDA Countries Without PRSP 22 Procurement5 Misprocurement 45 9 22 61 18 1 Procurement Complaints 448 475 301 Loan Administration6 Amendments/Extensions n.a. n.a. n.a. Refunds n.a. n.a. 5127 n.a.8 n.a. n.a. Cancellations 132 120 87 610.6 554.5 605.8 Loan Closings 137 137 127 Safeguards Projects subject to Corporate Review9 83 101 92 QAG Processes1011 QEA n.a. 124 n.a. QSA n.a. 69 n.a.


75

Table G.3: Category and Type of Key Controls by Audit Objective, System, and Frequency Distribution Across All Business Process Modules

AUDIT CATEGORY

Type 1 2 3 4 5 6 7 %Incidence 30% 4% 30% 30% 3% 1% 1%

Description 1. Validity/Existence: All transactions are properly authorized. 2. Segregation of Duties: Non-compatible control functions such as cash payment and cash authorization performed independently 3. Ownership: There are documented agreements in place to support the existence of transactions. 4. Completeness/Accuracy: All transactions are recorded properly. 5. Cut-Off: All transactions are in the correct reporting period. 6. Valuation: Loans, borrowings, and investment transactions are valued independently. 7. Disclosure: Adequate disclosure is made in the financial statements to comply with international financial report-ing standards.

TYPE AND SYSTEM Type/System Manual Automatic/Systemic Preventive Detective %Incidence 99% 1% 88% 12%

FREQUENCY Frequency Transactions Yearly Bi-Yearly Monthly Weekly Daily %Incidence 91% 4% 2% 0% 0% 3%

Table G.2: Links Identified by Management between Key Controls and the Five COSO Components, shown by Business Function

Module Function #

BPMs Control En-vironment

Risk As-sessment

Control Ac-tivities



Programming and Lending Products Internal Programming & Allocation 4 0 1 7 0 0 Lending Products 3 0 8 17 2 8

Fiduciary Services Related to Lending Financial Management 2 0 0 8 0 0 Loan Administration 10 0 0 32 4 0 Legal 4 0 0 24 15 2 Procurement 3 0 1 13 1 0 Safeguards 2 0 0 6 0 1

Quality Assurance QAG Processes 1 0 4 2 3 0 DISTRIBUTION OF COSO LINKS 29 0 14 109 25 11


76

Table G.5: Summary of Quality Ratings For Management’s Mapping and Assessment of Design Effectiveness of Key Controls

Quality Dimensions Average Ratings

Overall R1 R2 R3 R4 R5 Mapping the Business Process 2.38 Origin, Method, and Criteria 2.31 2.58 2.00 2.19 2.12 Accuracy and Completeness 2.50 2.31 2.73 2.00 2.42 Identification of Key Controls 2.19 2.38 2.04 2.08 Assessment of Control Design 2.62 Identifying Process Risks 2.81 2.46 4.00 2.31 2.15 Matching Risks with Process Design 2.31 2.35 2.08 2.44 Overall Average Rating 2.45 2.42 2.57 2.20 2.23 Rating Scale: 1 = Highly Satisfactory; 2 = Satisfactory; 3 = Satisfactory with Qualification; 4 = Less than Satisfactory.

TABLE G.4: SUMMARY OF AVERAGE QUALITY RATINGS OF MANAGEMENT’S ASSESSMENT OF THE DESIGN EFFECTIVENESS OF KEY CONTROLS

Mapping the Business Process Assessing Control Design Business Function

Overall

Origin, Method & Criteria

Accuracy & Completeness

ID of Key Controls Overall

ID of Process Risks

Match Risks

Overall Average

Programming & Allo-cation 2.50 2.00 3.00 2.00 3.00 2.50 3.00 2.57 Lending Products 2.33 2.00 2.00 2.33 2.67 2.67 2.33 2.33 Legal 2.25 2.25 2.25 2.25 2.50 3.00 2.00 2.36 Financial Manage-ment 2.00 2.00 2.00 2.00 2.00 2.50 2.00 2.07 Procurement 2.00 2.00 2.00 2.00 2.00 2.67 2.00 2.10 Loan Administration 2.67 2.67 3.00 2.33 2.89 3.00 2.56 2.73 Quality Assurance 3.00 3.00 3.00 2.00 3.00 3.00 2.00 2.71 Safeguards 2.00 2.00 2.00 2.00 2.50 2.50 2.00 2.14 Rating Scale: 1 = Highly Satisfactory; 2 = Satisfactory; 3 = Satisfactory with Qualification; 4 = Less than Satisfactory; N/A—Not Applicable.


77

Table G.6: Distribution of Quality Ratings Across Mapping and Control Design Dimensions

Distribution by Rating Mean 1 2 3 4 N Mapping the Business Process 2.38 0% 62% 38% 0% 26Origin, Method, and Criteria 2.31 0% 69% 31% 0% 26

Clarity of IDA Operational Objective? 2.58 4% 54% 23% 19% 26Clarity of method and criteria? 2.00 0% 100% 0% 0% 26BPM established under Bank BP or OP? 2.19 12% 62% 23% 4% 26Management sought input in process area? 2.12 0% 88% 12% 0% 26

Accuracy and Completeness 2.50 4% 46% 46% 4% 26Process has been clearly titled? 2.31 4% 62% 35% 0% 26Risks to BPM clearly stated? 2.73 4% 38% 38% 19% 26Ownership of Process clearly designated? 2.00 4% 92% 4% 0% 26Management sought input in process? 2.42 12% 50% 23% 15% 26

Identification of Key Controls 2.19 0% 81% 19% 0% 26Clear definition of key controls? 2.38 0% 69% 23% 8% 26Relevance of mapped BPM controls? 2.04 0% 96% 4% 0% 26Differentiation between controls for financial reporting

and other COSO objectives? 2.62 0% 92% 8% 0% 25

Assessment of Control Design 2.62 0% 38% 62% 0% 26Identifying Process Risks 2.46 0% 23% 73% 4% 26

Clear identification of risks that the control points are designed to alleviate? 2.46 8% 38% 54% 0% 26

Risks have been categorized (fin/op/rep) and analyzed? 4.00 0% 0% 0% 100% 26CTR documentation relates to the policies and procedures

of controls and risks? 2.31 0% 73% 23% 4% 26

CTR consulted with the most authoritative sources? 2.15 0% 85% 15% 0% 26Matching Risks with Process Design 2.31 0% 69% 31% 0% 26

Management adequately matched the design with the risks? 2.35 i. Built in checks and balances 2.81 0% 35% 50% 15% 26ii. Involved specialized staff 2.15 0% 85% 15% 0% 26iii. Involved appropriate operational units and mgmt levels? 2.08 0% 92% 8% 0% 26

Design process is known by relevant staff? 2.08 8% 81% 8% 4% 26Mgmt. has shown that controls extend to cover external risks? 2.44 0% 56% 44% 0% 25

Rating Scale: 1 = Highly Satisfactory; 2 = Satisfactory; 3 = Satisfactory with Qualification; 4 = Less than Satisfactory. Note: Modules 1-3 and Modules 25 & 26 were assessed together; therefore max. number of observations is 26 rather than 29.

All Modules Distribution by Rating 1 2 3 4

Distribution by No. 15 450 184 51 Distribution by Percentage 2% 64% 26% 7%


78

Table G.7: Distribution of BPMs According to Strategic Relevance and Risk Ranking Risk Categories*

1 2 3 Number Distribution

14 8 7 • IDA, FRM, & Post Conflict

Allocation • CAS Products • SIL—Project Cycle • DPL—Project Cycle • Contractual Remedies • SIL—Legal Regime • DPL—Legal Regime • FM—SIL • FM—DPL • SIL—Procurement • Procurement Complaints • Safeguards—SIL

• Corporate Review (ROC/OC)

• Procurement Non-Compliance

• Loan Management—SIL • Loan Management—DPL • LOA—Application Review • LOA—Suspensions • QAG—QAE and QSA • Safeguards—QACU

• Project Changes • LOA—Special Commitment • LOA—Amendment or Exten-

sion • LOA—Refund Process • LOA—Cancellation Process • LOA—Closings

(Standard & Special)

Average Quality Rating for Business Process Mapping 2.25 2.50 2.50

* STRATEGIC RELEVANCE AND RISK RANKING 1 = Highly Relevant, critical: heavy weight in management; major risks; high frequency of occurrence 2 = Relevant, but not critical: average weight in management; Some Risk; average frequency 3 = Relevant but not critical; moderate weight; moderate or minor risk; infrequent;


79

TABLE G.8: LISTING OF OPS AND BPS LINKED BY MANAGEMENT TO THE BPMS OP / BP # Operational & Bank Policies Covered OP/BP

1.00 OP Poverty Reduction 1.21* OMS Bank Financing of Recurrent Costs 2.11 BP Country Assistance Strategies X 2.20 OMS Project Appraisal 2.30 OP/BP Development Cooperation and Conflict

3.10 OP/BP Financial Terms and Conditions of IBRD Loans, IBRD Hedging Products, and IDA Credits X

4.00 OP/BP Use of Borrower Systems to Address Environ and Social Safeguard Issues

4.01 OP/BP Environmental Assessment X 4.02 OP/BP Environmental Action Plans 4.04 OP/BP Natural Habitats X 4.07 OP/BP Water Resource Management 4.09 OP Pest Management X 4.10 OP/BP Indigenous Peoples X 4.11 OP/BP Physical Cultural Resources X 4.12 OP/BP Involuntary Resettlement X 4.20 OP/BP Gender and Development 4.36 OP/BP Forests X 4.37 OP/BP Safety of Dams X 4.76 OP Tobacco 6.00 OP/BP Bank Financing X

6.30* OP/BP Local Costs Financing and Cost Sharing 6.60* OP/BP Financing of Interest during Construction

7.00 OP Lending Operations: Choice of Borrower and Contractual Agreement X

7.20 OP Security Arrangements 7.30 OP/BP Dealings with De Facto Governments

7.40 OP/BP Disputes over Defaults on External Debt, Expro-priation, and Breach of Contract

7.50 OP/BP Projects on International Waterways X 7.60 OP/BP Projects in Disputed Areas X 8.10 OP/BP Project Preparation Facility 8.30 OP/BP Financial Intermediary Lending 8.40 OP/BP Technical Assistance 8.45 OP/BP Grants 8.50 OP/BP Emergency Recovery Assistance X 8.60 OP/BP Development Policy Lending X

9.01 OD Procedures for Investment Operations under the Global Environment Facility

10.00 OP/BP Investment Lending: Identification to Board Pres-entation X


80

Listing of OPs and BPs Linked by Management to the BPMs

10.02 OP/BP Financial Management (OP rev. 4/04) X 10.04 OP/BP Economic Evaluation of Investment Ops 10.21 OP/BP Investment Operations Financed by the MFMP 10.70 OD Project Monitoring and Evaluation 11.00 OP/BP Procurement (OP rev. 4/04) X 12.00 OP/BP Disbursement X 12.10 OP Retroactive Financing X 12.20 OP/BP Special Accounts X 12.30 OP/BP Statement of Expenditures X

13.00 OP/BP Signing of Legal Documents and Effectiveness of Loans and Credits X

13.05 OP/BP Project Supervision X 13.16 OP/BP Country Portfolio Performance Reviews 13.20 OP/BP Additional Financing for Investment Lending X 13.25 OP/BP Use of Project Cost Savings (OP rev. 4/04) X 13.30 OP/BP Closing Dates X 13.40 OP/BP Suspension of Disbursements X 13.50 OP/BP Cancellations X 13.55 OP/BP Implementation Completion Reporting

13.60 OD Dissemination and Utilization of the OED Find-ings

14.10 OP/BP External Debt Reporting and Financial Statements X 14.20 OP/BP Cofinancing 14.25 OP/BP Guarantees 14.40 OP/BP Trust Funds

14.70 GP Involving Nongovernmental Organizations in Bank-Supported Activities

17.30 BP Comunications with Individual EDs

17.55 BP Inspection Panel Total 31 Percent 50% Note: * OPs/BPs and OMS are applied only to project in countries without approved country financing pa-rameters. BP—Bank Procedure, GP—Good Practice, OD—Operational Directive, OMS—Operational Memoranda, and OP—Operational Policy.

Total References in the 2006 World Bank Operational Manual Operational Policies and Bank Procedures (OPs & BPs) 56 Operational Directives (ODs) 3 Operational Manual Statement (OMS) 2 Good Practice (GPs) 1 Operational Memoranda (OPMs) 40 OP and BP Annexes 79 Total 181


81

NOTES 1. Source: Business Warehouse (BW)

2. Source: BW—Direct Costs by Service Report 4.1

3. Source: FRM and BW. Number of Post-Conflict countries by year: FY03 = 11, FY04 = 9, FY05=8.

4. Source: BW / OPCS

5. Source: OPCS

6. Source: Refunds—LOA; Cancellations—BW Loan Information Table; Closings—BW IEG Rat-ings table 4a1.

7. Of which 80% are SA/UN advance balance refund; other reasons 18%; World Bank error <2%

8. Data being collected, not available in published form

9. QACU and Regional EA Reviews of Safeguards A and B. Source: BW—Lending table 4a5

10. Source: QAG

83

Attachment 1: Management Report on its Review of IDA Controls—Part IA

Attachment 1

85

International Development Association

Management Report on Its Review of Internal Controls

Part IA

Prepared by: Operations Policy and Country Services Vice Presidency

Controllers, Strategy and Resource Management Vice Presidency October 6, 2006

86

TABLE OF CONTENTS

Introduction and Background ............................................................................................. 1 Methodology ........................................................................................................................ 1 Compliance with Applicable Laws and Regulations ........................................................... 2

Key Articles’ Provisions Governing IDA’s Operations.................................................................. 3

Main Policies Governing IDA’s Operations .................................................................................... 5

Business Processes and Key Associated Controls that Support Implementa-tion of the Principles and Procedures set out in Identified OP/BPs...................... 8

Management Findings and Recommendations ................................................................. 10 Annexes

1. Discussion of COSO and Methodology Used in the Assessment ........................................ 15

2. Table of Contents of Operational Manual ............................................................................... 19

3. IDA Lending by Instrument – FY05 and FY06........................................................................ 27

4. Business Processes Selected for Review................................................................................... 29

5. Sample of Process Flow Diagrams - Modules 4 and 8........................................................... 31

Attachment 1

87

LIST OF ACRONYMS BPs Bank Procedures – a component of the Bank’s Operational Manual CAS Country Assistance Strategy CFP Concessional Finance and Global Partnerships Vice Presidency COSO Committee of Sponsoring Organizations – issued an internal control framework CTR Controller and Vice President DPL Development Policy Lending operations IL Investment Lending operations IAD Internal Auditing Department IEG Independent Evaluation Group LEG Legal Vice Presidency OPCS Operations Policy and Country Services Vice Presidency OPs Operational Policies – a component of the Bank’s Operational Manual PBA Performance Based Allocation PMT Project Management Team SIL Specific Investment Loan SOX The Sarbanes-Oxley Act of 2002 PCAOB Public Company Accounting Oversight Board

Attachment 1

89

INTRODUCTION AND BACKGROUND

1. As reflected in the IDA 14 Replenishment Report1 Management committed to carry out, dur-ing the period of IDA 14, an independent comprehensive assessment of its control framework in-cluding internal controls over IDA operations and compliance with its charter and policies, and making such assessment available to the public after its disclosure has been approved by IDA’s Ex-ecutive Directors. This assessment was originally agreed to be completed by December 2005. However, due to the massive volume of work required, and unprecedented nature of this type of assessment, the original delivery date turned out to be unrealistic. 2. In furtherance of this commitment, Management met with the Board’s Audit Committee and Committee on Development Effectiveness (November 28, 2005) and again with the Audit Commit-tee (July 17, 2006). As explained by Management in these meetings and in the underlying docu-mentation, the comprehensive scope and lack of precedents for this type of assessment have made it necessary to carry out the overall assessment in two parts: Part I, compliance assessment (which in turn has been subdivided, as discussed with the Audit Committee in November, into Part IA, re-lating to design effectiveness of key controls, and Part IB, relating to operating effectiveness, or compliance with, such controls); and Part II, efficiency and effectiveness assessment. During the compliance assessment (Part I), Management is focusing on whether the existing internal control framework over IDA’s operations provides reasonable assurance that such operations are carried out in a manner that complies with the provisions of IDA’s charter (that is, IDA’s Articles of Agreement – referred to as the “Articles”) and internal policies governing IDA’s operations, with special focus on the mechanisms in place to ensure that funds are disbursed for the intended pur-poses. 3. This Report on Part IA sets out Management’s assessment relating to the design effectiveness of key controls currently in place to ensure compliance with the relevant Articles’ provisions and policies governing IDA's operations. It will be followed by a Report containing Management’s as-sessment of the operating effectiveness of, or compliance with, these controls.

METHODOLOGY

4. Management is conducting the assessment of IDA’s internal controls in the context of the COSO2 internal control framework. IDA adopted the COSO framework as its controls methodol-ogy in 1995. This framework is widely used by leading financial institutions in the United States and is also seen as a model in many other parts of the world. The COSO framework (described in more detail in Management’s Work Plans and in Annex 1) is an all encompassing process which covers all aspects of internal control of an organization’s operation. It considers not only the evaluation of formal controls, but also informal controls, such as ethics, trust, communication, or-

1 See, Report from the Executive Directors of the International Development Association to the Board of Governors,

Additions to IDA Resources: Fourteenth Replenishment, Working Together to Achieve the Millennium Develop-ment Goals, (approved by the Executive Directors of IDA on March 10, 2005), para. 39, under the Disclo-sure bullet.

2 COSO stands for the Committee of Sponsoring Organizations of the Treadway Commission, which pub-lished a report in 1992 titled “Internal Controls – Integrated – Framework.”

Attachment 1

90

ganization behavior and leadership, and incorporates “top-down” as well as “bottom-up” analysis. Like all modern control frameworks, the COSO framework requires that: (i) management assess the organization’s internal controls; and (ii) an independent third party verify such assessment. 5. Management determined that Part I of this exercise would be more valuable if carried out fol-lowing a “bottom up” approach. This exercise involves an unprecedented, comprehensive identifi-cation and cataloguing of the key business processes supporting the key internal controls and mechanisms throughout the operational complex of IDA. Management anticipates that this com-prehensive mapping will provide a solid foundation for the examination of institutional efficiency and effectiveness to be undertaken in Part II of this exercise. In carrying out the commitment made in the IDA 14 Replenishment Report, Management determined that it should not go into a lengthy, detailed historical analysis of IDA’s policies, but it should provide a timely, clear snapshot of IDA’s operational processes as practiced today. 6. As mentioned in paragraph 4, Management is carrying out its commitment in the context of the COSO framework. This framework rests on three pillars: (i) compliance with applicable laws and regulations; (ii) effectiveness and efficiency; and (iii) internal controls over financial reporting. Management already completed a comprehensive review of internal control over financial report-ing in line with industry best practice and has received an attestation from its external auditors on such controls since 1997. Management has initially focused additional efforts on the “compliance pillar”.

COMPLIANCE WITH APPLICABLE LAWS AND REGULATIONS

7. As an international organization established by an international treaty with privileges and immunities, IDA differs from the corporate entities for which the COSO framework was developed. As such, this assessment of IDA’s “compliance with laws and regulations” cannot focus on compli-ance with specific laws and regulations in various jurisdictions. Rather it must focus on whether the existing internal control framework over IDA’s operations provides reasonable assurance that such operations are carried out in a manner that complies with the provisions of the IDA’s Articles and internal policies governing IDA’s operations, including the mechanisms in place to ensure funds are disbursed for the intended purposes. 8. Any compliance assessment of internal controls over IDA’s operations must therefore go through a four-step process of:

a) identifying key provisions of the IDA Articles that govern IDA’s operations;

b) identifying main policies that were adopted by IDA to ensure that IDA’s operations are carried out consistently with these provisions;

c) identifying the manner in which these policies are intended to be carried out by cata-loguing the business processes and key controls put in place to ensure compliance with the identified policies and assessing the “design effectiveness” of these processes and key con-trols; and

d) assessing compliance with the business process and key controls by testing a sample of transactions.

Attachment 1

91

9. Management’s assessment relating to the first three steps is set out below. It is to be followed by a separate report relating to the compliance testing portion of the assessment. Key Articles’ Provisions Governing IDA’s Operations

10. Article I states that “the Association shall be guided in all its decisions by the provisions of [Article I]”. These provisions state:

“The purposes of the Association are to promote economic development, increase pro-ductivity and thus raise standards of living in the less-developed areas of the world in-cluded within the Association’s membership, in particular by providing finance to meet their important developmental requirements on terms which are more flexible and bear less heavily on the balance of payments than those of conventional loans, thereby fur-thering the developmental objectives of the International Bank of Reconstruction and Development (hereinafter called “the Bank”) and supplementing its activities.”

With these provisions in mind, Management has, for purposes of this exercise, identified the follow-ing key provisions set out in Article V, Operations, Section 1. Use of Resources and Conditions of Financ-ing, that serve as the foundation for carrying out IDA’s operations.

Purposes

a) Concessional Resources to Less Developed Areas

“V.1(a) The Association shall provide financing to further development in the less-developed areas of the world included within the Association’s membership.”

b) Financing High Development Priorities

“V.1(b) Financing provided by the Association shall be for purposes which in the opinion of the Association are of high developmental priority in the light of the needs of the area or areas concerned… ”

Project Related

c) Specific Projects and Special Circumstances Lending

“V.1(b) Financing provided by the Association .... except in special circumstances, shall be for specific projects.”

d) Use of Funds for Purposes Intended

“V.1(g) The Association shall make arrangements to ensure that the proceeds of any fi-nancing are used only for the purposes for which the financing was provided…“

e) Due Regard for Economy and Efficiency

“V.1(g) The Association shall make arrangements to ensure that the proceeds of any fi-nancing are used .... with due attention to considerations of economy, efficiency, and com-petitive international trade”

Attachment 1

92

f) Linking Disbursements to Expenditures as Incurred

“V.1(h) Funds to be provided under any financing operation shall be made available to the recipient only to meet expenses in connection with the project as they are actually in-curred.”

General

g) Lender of Last Resort

“V.1(c) The Association shall not provide financing if in its opinion such financing is available from private sources on terms which are reasonable for the recipient or could be provided by a loan of the type made by the Bank.”

h) Non-political Interference

“V.1(g) The Association shall make arrangement to ensure that the proceeds of any fi-nancing are used .... without regard to political or other non-economic influences or con-siderations.”3

For the purpose of this exercise, Article V, Operations, Section 2. Form and Terms of Financing, and Section 3. Modifications of Terms of Financing also set out important provisions, including: general financing provisions4; economic prospects5; acceptable borrower6; guarantees7; use of cur-rencies8; and modifications9.

3 See also Article V, Section 6 which states: “The Association and its officers shall not interfere in the po-

litical affairs of any member; nor shall they be influenced in their decisions by the political character of the member or members concerned. Only economic considerations shall be relevant to their decisions, and these considerations shall be weighted impartially in order to achieve the purposes stated in this Agreement.”

4 Article V, Section 2 (a) Financing by the Association shall take the form of loans, the Association may, however, provide other financing, either (i) out of funds subscribed pursuant to Article III, Section 1, and funds derived therefrom as principal, interest or other charges, if the authorization for such subscription expressly provides for such financing; or (ii) in special circumstances, out of supplementary resources furnished to the Association, and funds derived therefrom as principal, interest or other charges, if the ar-rangements under which such resources are furnished expressly authorize such financing.

5 Article V, Section 2 (b) Subject to the foregoing paragraph, the Association may provide financing in such forms and on such terms as it may deem appropriate, having regard to the economic position and prospects of the area or areas concerned and to the nature and requirements of the project.

6 Article V, Section 2 (c) Association may provide financing to a member, the government of a territory in-cluded within the Association’s membership, a political subdivision of any of the foregoing, a public, or private entity in the territories of a member or members, or to a public international or regional organiza-tion.

7 Article V, Section 2 (d) In the case of a loan to an entity other than a member, the Association may, in its discretion, require a suitable governmental or other guarantee or guarantees.

8 Article V, Section 2 (e) The Association, in special cases, may make foreign exchange available for local expenditures.

9 Article V, Section 3 The Association may, when and to the extent it deems appropriate in the light of all relevant circumstances, including the financial and economic situation and prospects of the member con-cerned, and on such conditions as it may determine, agree to a relaxation or other modification of the terms on which any of its financing shall have been provided.

Attachment 1

93

11. In identifying the above as the key Articles’ principles governing IDA’s operations, Manage-ment has focused on the gravity and impact that these provisions, as interpreted by the Executive Directors, have had on the manner in which IDA’s operations have been conducted since its incep-tion. In practice, other provisions of Article V have had less of an impact on day-to-day operations of IDA and therefore have not been focused on as part of this exercise.10 12. In order to reflect and comply with the above Articles’ principles, since its inception, IDA op-erations have been structured and implemented in a manner to ensure that scarce IDA resources were made available to support priority development activities in the poorest eligible member countries. The structure also meant that decisions regarding IDA’s financing would be made with-out regard to political or other non-economic influences or considerations and that IDA financing would be available for specific projects as well as other “special circumstances” operations. Across all modes of financing, IDA is required to ensure that its funds are used for the purposes intended to meet expenses as they are actually incurred, with due regard for economy and efficiency. Main Policies Governing IDA’s Operations

13. The policy framework governing IDA operations is the primary tool for setting the parame-ters for conducting IDA’s operations consistent with the above Articles’ principles. 14. The Executive Directors are responsible for interpreting IDA’s Articles and approving IDA’s operational policy framework, based on Management’s proposals. Over the years, to ensure that IDA’s operations are carried out in compliance with the Articles’ principles, while reflecting the evolving model underlying development assistance, the Executive Directors have approved various policies, which comprise the policy framework for IDA’s operations. These policies have been “translated” by Management into Operational Policies (OPs)11 and Bank Procedures (BPs)12 in-cluded in the Bank’s Operational Manual. The OPs and BPs put in place the main parameters and procedures to be followed by staff in conducting IDA’s operations in a manner that has been de-termined by Management and the Board to comply with the IDA Articles. 15. Currently, the Operational Manual contains over 100 OPs and BPs. (The table of contents of the Operational Manual is set out in Annex 2.) In order to determine which OP/BPs are most perti-nent to the key Articles’ principles identified above, Management focused on the three primary in-struments through which IDA currently conducts its operations:

a) Country Assistance Strategy (CAS), which sets out a country’s development priori-ties and the strategy for IDA support of such priorities over an agreed period of time.

10 One exception is Section 1 (d) of Article V, which requires that every IDA financing presented to the Ex-

ecutive Directors for approval be accompanied by a recommendation of a “competent committee”. This requirement is included in the business processes or flows for processing investment lending operations and development policy operations. However, over the years it has lost much of its control efficacy and has limited value in terms of risk mitigation.

11 OPs are short, focused statements that follow from the Articles of Agreement, the general conditions, and policies approved by the Board. OPs establish the parameters for the conduct of operations; they also de-scribe the circumstances under which exceptions to policy are admissible and spell out who authorizes exceptions.

12 BPs explain how staff carry out the policies set out in the OPs. They spell out the procedures and docu-mentation required to ensure Bankwide consistency and quality.

Attachment 1

94

b) Investment Lending Operations (IL), as represented by Specific Investment Loans (SILs), which are the primary tool for providing IDA financing for specific projects within the meaning of IDA’s Articles and to which IDA applies the requisite due diligence throughout the project cycle (from identification to completion) to ensure compliance, inter alia, with the key Articles’ principles.

c) Development Policy Lending Operations (DPL), which are the primary tool for pro-viding IDA financing for “special circumstances” operations within the meaning of IDA’s Articles and to which IDA applies the requisite due diligence throughout the project cycle (from identification to completion) to ensure compliance, inter alia, with the key Articles’ principles.

16. Management assessment focused on these three primary instruments, and the policies and procedures that apply to them, as a result of its determination that the three represented a signifi-cant portion of IDA operations in terms of the overall numbers and volume of IDA yearly new commitments. In FY05 and FY06 combined, IDA approved a total of 327 operations totaling $17.4 billion. Of these, 263 operations or $12.6 billion were IL operations, 62 and $4.6 billion were DPL operations and 2 were Guarantee operations. Further, of the 263 IL operations, 163 (or 62%) were SIL operations, accounting for $8.0 billion of the $17.4 billion. Overall, DPL and SIL operations ac-counted for 72% or $12.6 billion of the $17.4 billion. (See Annex 3 for more details). Similar propor-tions are reflected when looking at the “active” IDA portfolio under implementation. 17. To reflect the above, in identifying key polices governing IDA’s operations, Management fo-cused on “flagship” OP/BPs that are most critical to the three primary instruments and compliance with the key Articles’ principles. These “flagship” OP/BPs include four umbrella statements gov-erning financial terms of and eligibility for IDA financing as well as policies and procedures gov-erning the three primary instruments for carrying out IDA operations, namely:

a) Umbrella statement governing financial terms of and eligibility for IDA financing

OP/BP 3.10, Financial Terms and Conditions of IBRD Loan, IBRD Hedging Products, and IDA Credits, which sets out the financial terms and conditions of IDA Credits.

b) Umbrella statement governing CASs

OP/BP 2.11, Country Assistance Strategies, which sets out the key processes that apply to the CAS product from identification through completion.

c) Umbrella statements governing ILs

OP/BP 10.00, Investment Lending: Identification to Board Presentation, which sets out the parameters for processing investment projects from identification through Board presenta-tion.

OP/BP 13.05, Project Supervision, which sets out the parameters for supervision of in-vestment projects under implementation.

d) Umbrella statement governing DPLs

OP/BP 8.60, Development Policy Lending, which sets out the parameters for processing of DPLs from identification through completion.

Attachment 1

95

18. In addition to these four umbrella statements, the Operational Manual also includes specific policies governing fiduciary, contractual and safeguards requirements and associated procedures to be followed when IDA finances either IL or DPL operations. These specific policies are:

a) Policies and procedures governing financial management aspects (OP/BP 10.02, Fi-nancial Management).

b) Policies and procedures governing disbursement aspects (OP/BP 12.00, Disburse-ment, 12.20, Special Accounts, OP/BP 12.30, Statements of Expenditure, OP/BP 6.00, Bank Financing (Expenditure Eligibility).

c) Polices and procedures governing procurement aspects (OP/BP 11.00, Procurement).

d) Policies and procedures governing contractual/legal aspects (OP/BP 7.00, Lending Operations: Choice of Borrower and Contractual Agreements, OP/BP 13.00, Signing of Legal Documents and Effectiveness of Loans and Credits, OP/BP 13.30, Closing Dates, OP/BP 13.40, Suspension of Disbursements, OP/BP 13.50, Cancellations, OP/BP 14.10, External Debt Reporting and Financial Statements).

e) Policies and procedures governing safeguards aspects (OP/BP 4.01, Environmental Assessment, OP/BP 4.04, Natural Habitats, OP 4.09, Pest Management, OP/BP 4.10, In-digenous Peoples, OP/BP 4.11, Physical Cultural Resources, OP/BP. 4.12, Involuntary Resettlement, OP/BP 4.36, Forests, OP/BP 4.37, Safety of Dams, OP/BP 7.50, Projects on International Waterways, OP/BP 7.60, Projects in Disputed Areas).

19. The table of contents of the Operational Manual set out in Annex 2 highlights the OP/BPs that were specifically looked at as part of this assessment. The OPs and BPs that were not specifically mapped by Management as part of this exercise can be classified into the following broad catego-ries:

• OP/BPs governing various trust funds and grants that do not utilize IDA re-sources (e.g., OP/BP 8.45; OD 9.01; OP/BP 10.21; OP/BP 14.20; OP/BP 14.40);

• Ops/BPs that apply to specific other types of investment lending products (Emergency Recovery Credits (OP/BP 8.50), Financial Intermediary Credits (OP/BP 8.30) and Techni-cal Assistance Credits (OP/BP 8.40)), given that the SIL was deemed to be a representative proxy for all investment lending in terms of volume and main processes;

• OP/BP 14.25 governing guarantees, given the very small volume of guarantees in the IDA portfolio;

• Thematic and content-specific OP/BPs that feed into the processes that were mapped as part of this exercise, but do not have separate processes and controls to be captured in this phase of this exercise (e.g., OP/BP 1.00, 4.07, OP/BP 4.20, OMS 2.20, OP/BP 10.04; 6.00 (which supersedes OP/BP 6.30, 6.60) OP/BP 11.03); and

• OP/BPs relating to specialized contractual and other issues which, when they arise, are addressed as part of the processes and controls that have been mapped (e.g., OP/BP 2.30, OP 7.20, OP/BP 7.30, 7.40, 7.50, 7.60).

Attachment 1

96

Business Processes and Key Associated Controls that Support Implementation of the Princi-ples and Procedures set out in Identified OP/BPs 20. In assessing how staff comply with the above policies and procedures, Management identified the relevant business processes in place which staff are expected to use as guidance and best prac-tice when working on IDA operations. For the purposes of this exercise, Management, in consulta-tion and agreement with the Internal Auditing Department (IAD) and the Independent Evaluation Group (IEG), focused on the period from July 1, 2005, through March 31, 2006, (the period under review). These business process reviews are the tool that Management has used to document its processes, risks and controls related to the preparation of the financial statements, and this tool is also being used in documenting the controls over IDA’s compliance with its Articles and applicable internal policies and procedures. These business process reviews were performed through a com-bination of process flows and narratives, and risk assessments. The total number of 27 process overviews (process #30 – Debt Sustainability Analysis has not yet been completed and is not in-cluded), and detailed descriptions for 108 key controls when set out in narrative form give rise to a document of about 350 pages. (See Annex 5 for examples of process flow charts.) These process flows and narratives were prepared by the staff of the Project Management Team (PMT), with input from individual subject matter experts of the identified processes in OPCS and other units as re-quired. Management believes that all these identified activities and controls have been accurately reflected. 21. For the purposes of this Report, Management’s assessment of the design effectiveness of con-trols associated with the business processes identified was focused on addressing whether the sys-tem of such internal controls is both comprehensive as well as suitably designed to prevent or de-tect on a timely basis, material issues of non-compliance or significant control deficiencies. Key controls have been evaluated for their design effectiveness by reviewing the process maps and as-sociated key controls documentation to ensure that internal controls have been suitably designed. The key controls as defined by the PMT and/or the specific subject matter experts within Control-ler’s and Operations Policy and Country Services (OPCS) have been corroborated by (a) examina-tion of documentary evidence, (b) observation, and/or (c) re-performance. This process has led to the production of detailed flow charts relating to 27 of the 28 main business processes (process #30 – Debt Sustainability Analysis has not yet been completed and is not included) and key associated controls. (The list of the 28 main business processes is set out in Annex 4.) 22. Management’s decision relating to identification and grouping of these 28 main business processes and key controls associated with them mirrors the manner in which Management identi-fied and grouped the policies that apply to the three primary IDA instruments as set out above. Specifically, in support of the four umbrella statements, which govern financial terms of and eligi-bility for IDA financing as well as the three primary instruments for carrying out IDA operations, Management has identified and mapped, or catalogued, the following four umbrella processes:

a) Processes that apply to IDA allocation13 (Nos.1 and 30, See Annex 4). 13 Please note that IDA's allocations are carried out in accordance with the IDA Performance Based

Allocation System (PBA) -- a set of rules by which IDA allocates its resources to each IDA recipient. With modifications, this has been used since 1977 (IDA 5). The PBA rules and criteria have been adopted over successive IDA replenishments and are contained in the sequence of replenishment re-ports.

Attachment 1

97

b) Processes that apply to the CAS from identification through completion (No. 4, see Annex 4).

c) Processes that apply to SILs from identification through completion (No. 5, see Annex 4).

d) Processes that apply to DPLs from identification through completion (No. 7, see Annex 4).

23. Consistent with Management’s grouping of the relevant policies as set out above, Manage-ment then identified and mapped business processes that support implementation of the specific policies governing fiduciary, contractual and safeguards requirements and associated procedures relating to CAS, SIL and DPL, all of which feed into the four umbrella business processes identified in paragraph 17 above. These include:

a) Processes relating to corporate review with respect to CAS, SIL and DPL (No. 8, see Annex 4).

b) Processes relating to implementation of policies and procedures governing financial management aspects (Nos. 12 and 13, relating to OP/BP 10.02, Financial Management, see Annex 4).

c) Processes relating to implementation of policies and procedures governing disburse-ment aspects (Nos. 17-26 relating to OP/BP 12.00, Disbursement, OP/BP 12.10, Retroac-tive Financing, OP/BP 12.20, Special Accounts, OP/BP 12.30, Statements of Expenditure, see Annex 4).

d) Processes relating to implementation of polices and procedures governing procure-ment aspects (Nos. 14-16, relating to OP/BP 11.00, Procurement, see Annex 4).

e) Processes relating to implementation of policies and procedures governing contrac-tual/legal aspects (Nos. 6, 9-11, relating (OP/BP 7.00, Lending Operations: Choice of Bor-rower and Contractual Agreements, OP/BP 13.00, Signing of Legal Documents and Effec-tiveness of Loans and Credits, OP/BP 13.30, Closing Dates, OP/BP 13.40, Suspension of Disbursements, OP/BP 13.50, Cancellations, OP/BP 14.10, External Debt Reporting and Financial Statements, see Annex 4).

f) Processes relating to implementation of policies and procedures governing safeguards as-pects (Nos. 28 and 29, relating (OP/BP 4.01, Environmental Assessment, OP/BP 4.04, Natural Habitats, OP/BP 4.09, Pest Management, OP/BP 4.10, Indigenous Peoples, OP/BP 4.11, Physical Cultural Resources, OP/BP. 4.12, Involuntary Resettlement, OP 4.36, Forests, OP/BP 4.37, Safety of Dams, see Annex 4).

24. In addition to the above, Management also mapped the business processes that apply to the quality assurance function which, while not mandated by any specific policy, has played an impor-tant role in reviewing quality aspects of operational work both at appraisal and at supervision stages (No. 27, see Annex 4).

Attachment 1

98

Management Findings and Recommendations

25. The design effectiveness assessment captured a representative picture of the control environ-ment over IDA’s operations at the transaction level, where daily decisions are made which have a direct impact on the use of IDA resources. While this turned out to be an extremely involved and labor intensive exercise for which there was little precedent inside or outside the institution, Man-agement found it to be valuable. Management expects that the information gleaned as a result of this exercise will provide tangible support for strategic and rationalization decisions for IDA's op-erations going forward. 26. On the basis of the above-described methodical, interactive and thorough exercise of perform-ing a bottom up identification and cataloguing of the processes and associated controls for carrying out IDA’s operations, Management has the findings and recommendations set out below.

A. The examination of the key policies and procedures that govern IDA operations and the mapping and review of the key business processes and associated controls that enable compliance with these policies confirmed that:

• The performance based allocation model is appropriately designed to direct scarce IDA resources in support of priority development activities in the poorest eligible member countries.

• The three primary instruments for carrying out IDA operations (i.e, CAS, SIL and DPL), their complementary use and the processes and controls that apply to them from identification to completion have been appropriately designed to verify that:

o IDA financing is being provided in support of developmental priorities and is focused on matters that appropriately fall within IDA’s mandate

o Consistent with the Articles’ provisions, IDA financing is made available for specific projects as well as other “special circumstances” operations where appropriate.

• The umbrella processes and associated controls for processing CAS, SIL and DPL through all stages (from identification to completion) and the specific processes and controls that fall under these umbrella processes (fiduciary, contractual, safeguards, etc.) are appropriately designed to facilitate and verify compliance with the key IDA policies and procedures adopted to ensure that IDA funds are used for the purposes intended to meet expenses as they are actually incurred.

The specific processes and associated controls related to procurement are appropriately de-signed to facilitate and verify compliance with IDA’s procurement policies, as reflected in OP/BP 11.00 and Guidelines: Procurement under IBRD Loans and IDA Credits and Guidelines: Selec-tion and Employment of Consultants by World Bank Borrowers and meet the objective of using IDA resources to finance goods, works and services that were procured by the borrower with due regard for economy and efficiency.

B. Management had difficulties with obtaining timely access to relevant documents that are needed to carry out the compliance testing portion of the assessment. This work clearly revealed that there is an issue with respect to document retention and accessibility which merits serious examination and improvement. Although the auditors retained by Management to do compliance testing informed Management that document retention

Attachment 1

99

and accessibility is a common problem in many corporations that have transitioned from manual retention and filing of hard copies of documents to filing and archiving docu-ments electronically, Management intends to address the document retention and acces-sibility issue immediately by setting up an expert panel to look at retention, filing and accessibility of operational documents and come up with recommendations for im-provements within the next six months.

C. Management has also identified that the OPs and BPs included in the Operational Manual are not keeping pace with the changes on the ground that are being introduced from time to time. One example is BP 10.00 on processing investment lending from iden-tification to Board approval, which is in need of urgent update. Other examples in need of updating to reflect current requirements which have been approved by the Board over the past several years include the 12.00 series governing disbursements, and OP/BP 10.02 on financial management. As part of Management’s assessment of the effectiveness and efficiency of IDA’s internal controls framework, Management intends to look at the current processes underlying policy revision to determine if they need to be revised to facilitate more efficient and timely updating of operational policies and procedures.

D. The policy framework governing IL operations is too complex and disjointed, mak-ing it hard for staff to identify all the policies with which they are expected to comply when working on IL operations. Following the example of the recent reform of the DPL policy and procedures (OP/BP 8.60), Management has been actively working on ration-alization and consolidation of policies governing IL processing from identification to completion.

E. The assessment confirmed that many staff find the existing processes and documen-tary requirements very onerous and inefficient. To address this problem, at least in part, Management intends to issue in the next few months standard updated operational templates to be used by staff in documenting various steps in the IL and DPL proc-esses.

F. The assessment identified that there is a disparity in the frequency of corporate reviews of SIL and DPL operations, with all DPLs being subject to such review and relatively few SILs. While some of the heightened scrutiny of the DPLs is consistent with the Articles’ provisions relating to “special circumstances” lending, Management is examining whether there is a need to review criteria for submitting Investment Lending opera-tions that raise special risks or issues to the corporate review process.

G. As part of the detail review of key controls and process flows (i.e., “walkthroughs”) performed with respect to the individual business processes, Management has identified a number of areas that merit a closer look. Some of these areas are:

1. Several existing policies and procedures need to be updated or enhanced or, in some instances, additional guidance needs to be introduced.

2. Certain system capabilities or system-related controls need to be better aligned with the process requirements.

3. Timeliness of processes related to managing individual credits should be improved.

4. Variances in regional implementation of institutionally endorsed guidelines need to be reviewed to ascertain whether these variances are appropriate.

Attachment 1

100

5. Certain procurement processes and controls require enhancements to strengthen ef-fectiveness.

6. Processes surrounding SILs with regard to project changes or contractual remedies need to be strengthened.

7. Clarification should be issued to require staff, which provide conditional clearances or feedback on project documents, follow up to ensure that their comments have been incorporated, as appropriate.

Management intends to closely examine these areas as part of the compliance phase of the project and to develop corrective action plans for the findings that require remedia-tion.

Attachment 1

101

ANNEX 1: DISCUSSION OF COSO AND METHODOLOGY USED IN THE ASSESSMENT

1. In performing its review of the compliance with IDA’s Articles and applicable internal policies and procedures, Management has used, as much as possible, the concepts as defined in the Audit-ing Standard No. 2 (AS2) An Audit of Internal Control Over Financial Reporting Performed in Conjunc-tion with An Audit of Financial Statements issued by the U.S. Public Company Accounting Oversight Board (PCAOB) in response to the provisions of Section 404 of the U.S. Sarbanes-Oxley Act of 2002 (SOX legislation). The Bank is currently performing its assessment of internal controls over external financial reporting using existing auditing standards on attestation of internal controls over finan-cial reporting as prescribed by generally accepted auditing standards. Applying these concepts, which have been defined by standard setters for assessing internal controls over financial reporting, provides Management a level of comprehensiveness, rigor and standardization required in its as-sessment of internal controls and compliance. 2. Under the COSO framework, the effectiveness of an internal control system is measured by its capacity to provide reasonable assurance to Management and the Executive Directors (the Board) regarding the achievement of the Bank’s objectives in the following three areas (pillars):

• Reliability of financial reporting – relating to preparation of published financial state-ments;

• Compliance with applicable laws and regulations – relating to compliance with appli-cable legal and regulatory framework (charter, policies, etc.); and

• Effectiveness and efficiency of operations – relating to effective and efficient use of re-sources.

3. COSO’s categorization allows focusing on separate aspects of internal controls, addressing dif-ferent needs and taking into account that each area may be the direct responsibility of different or-ganizational units and managers. As briefly summarized below, this categorization also distin-guishes between what can be expected from each category of internal control. 4. Financial Reporting: Bank/IDA’s primary objective in adopting the COSO framework was to focus on financial reporting, which is the primary focus of the Bank’s external auditor. Since 1997, Management has annually asserted that it maintains an adequate system of internal control over ex-ternal financial reporting for both IBRD and IDA; and Management has received an attestation to this effect from the external auditor at the end of each fiscal year.1

1 The examination of financial reporting under the COSO framework has also enabled the Bank to be

well positioned in reviewing the impact of various shareholder nation’s legislation related to corporate governance requirements (i.e. the U.S. Sarbanes-Oxley Act of 2002 (SOX)). SOX is a set of legislative re-forms adopted in the United States to strengthen corporate governance and regulatory activities for fi-nancial reporting by capital market participants, and the EU 8th Directive (Proposal for a Directive of the European Parliament and the Council on Statutory Audit of Annual and Consolidated Accounts and Amending Council Directives, issued on March 16, 2004) and enhancing its internal controls, where appropriate.

Attachment 1

102

5. Effectiveness and Efficiency of Operations: Unlike the areas of financial reporting and compli-ance with laws and regulations, which largely depend on performance of activities within the organi-zation’s control, effectiveness and efficiency of operations depend on a host of factors that are often outside the organization’s control and over which Bank/IDA staff and management may have little influence. Specifically:

• Internal controls cannot eliminate bad judgment or decisions, nor can they influence external events that may result in a failure to achieve operations’ goals or objectives; and

• With respect to these objectives, the internal control system can provide reasonable assurance only that Management and, in its oversight role, the Board are made aware, in a timely manner, of the extent to which the organization is moving towards meet-ing the agreed goals or objectives.

6. With the above caveats, as part of this exercise, Management intends to assess whether the ex-isting internal control framework over IDA’s operations provides reasonable assurance that IDA’s operations are carried out efficiently and effectively. 7. The review of internal controls for IDA’s operations are being limited to the areas of “compli-ance with applicable laws and regulations” and “effectiveness and efficiency of operations” under the COSO framework. Given that the financial reporting controls are assessed annually by Man-agement and verified by the external auditors as part of the annual audit, no additional assessment in this area was believed necessary in the context of the overall internal control assessment. Division of Responsibilities 8. Management’s Assessment of IDA’s Internal Controls – Consistent with the COSO framework requirements Management will assess the key internal controls for IDA operations in the areas of compliance and effectiveness and efficiency of operations. This assessment includes the identifica-tion of and compliance with the relevant controls (Parts I(A) and I(B)) and a review and assessment of the efficiency and effectiveness of their design and operation (Part II). 9. The assessment is being led by the Offices of the Vice President and Controller (CTR), and the Vice President, Operational Policy and Country Services (OPCS), with key support roles for Con-cessional Finance and Global Partnerships (CFP) and Legal (LEG). To manage the implementation of this process, Management has created a high-level Steering Committee comprised of the Vice Presidents of the above four units with the inclusion of the Internal Audit Department (IAD) and the Independent Evaluation Group (IEG) management as observers. The responsibilities of the Steering Committee are to:

• Define the objectives of the assessment in the areas of compliance and efficiency and effectiveness of operations;

• Oversee the implementation of the assessment and coordinate Management’s activi-ties with those of IAD and IEG;

Attachment 1

103

• Review the controls issues identified during the assessment and evaluate whether these issues rise to the level of material weaknesses that would impact Management’s assessment; and

• Review and endorse Management’s assessment and/or report of its review prior to its transmittal to IAD, IEG and the President.

10. To assist the Steering Committee in implementing the detailed work program, a Project Man-agement Team (PMT) consisting of the Directors of Accounting and OPCS’s Country Services De-partments and certain senior staff within these units has been established. The responsibilities of the PMT include:

• Preparing the detailed scoping and implementation plan and obtaining its approval by the Steering Committee;

• Bring all the respective units together in the Bank to brief them as to the work pro-gram and where their assistance will be required;

• Documenting the risks, processes and related controls associated with ensuring IDA’s funds are being used for the purposes described in the credit agreements;

• Performing the assessment of the design and operational effectiveness of the con-trols;

• Identifying instances of non-compliance with prescribed controls and related control deficiencies;

• Evaluating whether the identified control deficiencies, individually or in aggregate, have a material impact on Management’s ability to state whether internal controls are adequate over compliance with IDA’s charter and applicable internal policies and procedures; and

• Preparing Management’s assessment and/or report of its review for review and en-dorsement by the Steering Committee; and

• Liaising with IAD and IEG in the preparation of their respective validation and evaluation.

11. IAD Review of Management’s Assessment – Consistent with IAD’s overall mandate, Man-agement’s assessment of internal controls over IDA’s operations in the areas of compliance (Parts I(A) and I(B)) and efficiency and effectiveness (Part II) will be reviewed by IAD. 12. Overall IEG Evaluation – in order to fulfill Management’s commitment of carrying out an in-dependent assessment of its internal controls over IDA operations, as specified in the IDA 14 Report, IEG will evaluate (i) the work performed and findings identified by Management, in the areas of compliance (Parts I(A) and I(B)) and efficiency and effectiveness (Part II) of the internal controls framework over IDA operations, as well as (ii) IAD’s review of Management’s assessment (includ-ing methodology, testing results, and reported findings). Based on these evaluations, IEG would then issue its report. [If the timing permits, IEG would also contribute a section to the report on Part IA, in time for the meeting of the IDA Deputies in late November.]

Attachment 1

104

ANNEX 2: TABLE OF CONTENTS OF OPERATIONAL MANUAL

Operational Manual: Table of Contents

VOLUME I: STRATEGIES AND PRODUCTS Country Focus

1.00 - Poverty Reduction OP 2.11 - Country Assistance Strategies BP 2.30 - Development Cooperation and Conflict OP BP 13.16 - Country Portfolio Performance Reviews OP BP Sector/Thematic Strategies 4.02 - Environmental Action Plans OP BP 4.07 - Water Resources Management OP 4.20 - Gender and Development OP BP Business Products and Instruments 8.10 - Project Preparation Facility OP BP 8.30 - Financial Intermediary Lending OP BP 8.40 - Technical Assistance OP BP Annex A-Contents of a Technical Annex to the Memorandum and Recommendation of

the President

Attachment 1

105

8.45 - Grants OP Annex A-DGF Eligibility Criteria BP 8.50 - Emergency Recovery Assistance OP BP Annex A-Content of the Technical Annex to the Memorandum and Recommendation of

the President (MOP) for Emergency Recovery Loans 8.60 - Development Policy Lending OP BP 9.01 - Procedures for Investment Operations under the Global Environment Facility OD Annex A-Procedures for Global Environmental Facility Investment Operations--

Guidelines for Executive Project Summary Annex C1-Sample Letter of Agreement for a Project Preparation Advance from the

Global Environment Trust Fund Annex D-Global Environment Facility Investment Operations--Guidelines for the Memo-

randum and Recommendation of the Director Annex D1-Guidelines for Preparing the "Grant and Project Summary" for the MOD 10.00 - Investment Lending: Identification to Board Presentation OP BP Annex A-Outline for an Investment Project Information Document Annex B-Elements of a Project Implementation Plan Annex C-Operational Plan Contents Annex D-Outline for a Staff Appraisal Report Annex D1-Outline for Preparing the "Loan and Project Summary" Annex D2-Standard Disbursement Profiles Annex E-Outline for a Memorandum and Recommendation of the President Annex F-Sample Notice of Invitation to Negotiate Annex G-Telex of Invitation to Negotiate Annex H-Notice of Status of Negotiations Annex I-Loan/Credit/GEF Grant Cover Sheet Annex J-Conditions of Board Presentation Annex K-Streamlined Procedures for Board Presentation

Attachment 1

106

10.21 -Investment Operations Financed by the Multilateral Fund for the Implementation of the Montreal Protocol

OP Annex A-The Montreal Protocol Annex B-Eligible Activities BP Annex A-Project Preparation Advances from the Ozone Projects Trust Fund Annex A1-Sample Letter of Agreement for a Project Preparation Advance from the

Ozone Projects Trust Fund Annex B-Montreal Protocol Operations: Outline for the Project Information Document Annex C-Presentation of Projects to the MFMP Executive Committee Annex C1-Sample Project Cover Sheet and Summary Annex D-Outline for Memorandum and Recommendation of the Director Annex E-Sample Notice to Prospective Recipients of Grants under the Multilateral

Fund for the Implementation of the Montreal Protocol Annex F-Disbursement under Ozone Projects Trust Fund Grant Agreements 14.25 - Guarantees OP BP Partnerships 14.20 - Cofinancing OP Annex A-Sources and Types of Cofinancing BP 14.40 - Trust Funds OP BP 14.70 - Involving Nongovernmental Organizations in Bank-Supported Activities GP VOLUME II: PROJECT REQUIREMENTS Safeguard Policies 4.00 - Piloting the Use of Borrower Systems to Address Environmental and Social Safeguard Is-

sues in Bank-Supported Projects OP Table A1-Environmental and Social Safeguard Policies—Policy Objectives and Operational

Principles BP

Attachment 1

107

4.01 - Environmental Assessment OP Annex A-Definitions Annex B-Content of an Environmental Assessment Report for a Category A Project Annex C-Environmental Management Plan BP Annex A-Environmental Data Sheet for Projects in the IBRD/IDA Lending Program Annex B-Application of EA to Dam and Reservoir Projects Annex C-Application of EA to Projects Involving Pest Management 4.04 - Natural Habitats OP Annex A-Definitions BP 4.09 - Pest Management OP 4.10 - Indigenous Peoples OP Annex A-Social Assessment Annex B-Indigenous Peoples Plans Annex C-Indigenous Peoples Planning Framework BP 4.11 – Physical Cultural Resources OP BP 4.12 - Involuntary Resettlement OP Annex A-Involuntary Resettlement Instruments BP 4.36 - Forests OP Annex A-Definitions BP 4.37 - Safety of Dams OP BP Annex A-Dam Safety Reports: Content and Timing 7.50 - Projects on International Waterways OP BP 7.60 - Projects in Disputed Areas OP BP

Attachment 1

108

11.03 - Management of Cultural Property in Bank-Financed Projects OPN Analysis 10.04 - Economic Evaluation of Investment Operations OP BP Fiduciary 4.76 - Tobacco OP 10.02 - Financial Management OP BP Annex A-Review of Financial Management Systems Annex B-Sample Telexes: Accounting; Financial Reporting; and Auditing Annex C-Audit Reports Compliance System 11.00 - Procurement OP BP Annex A-The World Bank Procurement Function 12.00 - Disbursement OP BP Annex A-Valuation of Disbursements and Changes in Exchange Rates 12.20 - Special Accounts OP Annex A-Required Bank Characteristics Annex A1-Sample Comfort Letter from Commercial Bank Holding Special Account Annex B-Subaccounts and Second-Generation Special Accounts BP 12.30 - Statements of Expenditure OP BP Financial 6.00 - Bank Financing OP BP Annex A-Special Authorization Arrangements for Selected Types of Expenditure

Attachment 1

109

6.30 - Local Cost Financing and Cost Sharing OP Annex A-Definitions Annex B-Calculation of a Project's Foreign Exchange Costs BP 6.60 - Financing of Interest during Construction OP BP 12.10 - Retroactive Financing OP 13.20 - Additional Financing for Investment Lending OP BP 13.25 - Use of Project Cost Savings OP BP Management 10.70 - Project Monitoring and Evaluation OD Annex A-List of Publications Available on Project Monitoring and Evaluation 13.05 - Project Supervision OP BP 13.55 - Implementation Completion Reporting OP BP 13.60 - Dissemination and Utilization of the Operations Evaluation Department (OED) Findings OD 17.30 - Communications with Individual Executive Directors BP 17.55 - Inspection Panel BP Annex A-Inspection Panel Resolution Annex B-Review of the Resolution Establishing the Inspection Panel: Clarification of

Certain Aspects of the Resolution Annex C-Conclusions of the Board's Second Review of the Inspection Panel

Attachment 1

110

Contractual 3.10 - Financial Terms and Conditions of IBRD Loans, IBRD Hedging Products, and IDA Cred-

its OP Annex A-Past Loans of IBRD Annex B-Prepayment of IBRD Loans Annex C-Countries Ranked by Per Capita Income Annex D-IBRD/IDA Countries: Per Capita Incomes, Lending Eligibility, and Repayment

Terms BP 7.00 - Lending Operations: Choice of Borrower and Contractual Agreements OP 7.20 - Security Arrangements OP 7.30 - Dealings with De Facto Governments OP BP 7.40 - Disputes over Defaults on External Debt, Expropriation, and Breach of Contract OP BP 13.00 - Signing of Legal Documents and Effectiveness of Loans and Credits OP BP 13.30 - Closing Dates OP BP 13.40 - Suspension of Disbursements OP BP Annex A-Sample Notice to the Borrower for a Suspension Unrelated to Payment Annex B-Sample Notice to the Executive Directors for a Suspension Unrelated to Pay-

ment Annex C-Sample Notice of Cancellation Sent to the Borrower Annex D-Sample Notice of Cancellation Sent to the Executive Directors Annex E-Sample Notice to the Borrower for Lifting a Suspension Annex F-Sample Notice Sent to the Executive Directors for Lifting a Suspension Unre-

lated to Payment Annex G-Sample Notice to the Borrower When Payment Is 30 Days Overdue Annex H-Sample Notice of Impending Suspension Sent to the Borrower When Payment

Is 45 Days Overdue Annex I-Sample Notice of Suspension Sent to the Borrower When Payment Is 60 Days

Overdue Annex J-Sample Notice of Suspension Sent to the Executive Directors for a Payment-

Related Suspension

Attachment 1

111

Annex K-Sample Notice to the Executive Directors for Lifting a Payment- Related Sus-pension

13.50 - Cancellations OP BP 14.10 - External Debt Reporting and Financial Statements OP BP Annex A-Sample Letter on Financial and Economic Data: IBRD Borrowing Country Annex B-Sample Letter on Financial and Economic Data: IDA Borrowing Country Annex C-Sample Letter of Representations regarding a Borrower's/Project Entity's Fi-

nancial Condition Other - The World Bank Policy on Disclosure of Information OP

Attachment 1

112

ANNEX 3: IDA LENDING BY INSTRUMENT FY05 AND FY06

The following table represents the IDA lending by type of major lending instrument for the fiscal years 2005 and 2006.

No.% of Total

Amt (US$M)

% of Total No.

% of Total

Amt (US$M)

% of Total No.

% of Total

Amt (US$M)

% of Total

DPL 32 20.0% 2.16 27.3% 30 18.0% 2.42 25.5% 62 19.0% 4.59 26.3%

Investment Lending 126 78.8% 5.63 71.0% 137 82.0% 7.02 73.9% 263 80.4% 12.65 72.6%

SIL 77 48.1% 3.31 41.8% 86 51.5% 4.71 49.5% 163 49.8% 8.02 46.0%

Other Investment 49 30.6% 2.31 29.2% 51 30.5% 2.32 24.3% 100 30.6% 4.63 26.6%

Guarantees 2 1.3% 0.14 1.7% 0 0.0% 0.06 0.6% 2 0.6% 0.20 1.1%

Total 160 100.0% 7.92 100.0% 167 100.0% 9.51 100.0% 327 100.0% 17.43 100.0%

Combined ApprovalsProjects Committed

IDA Lending

Projects Committed Projects CommittedLending

Instrument

Approved FY05 Approved FY06

Attachment 1

113

ANNEX 4: BUSINESS PROCESSES SELECTED FOR REVIEW

# List of IDA Business Processes Reviewed 1 IDA Allocation

IDA Allocation Model *

IDA Post Conflict Allocation *

4 Country Assistance Strategy: preparation and processing of country strategy documents (CAS etc.)

5 Specific Investment Loan (SIL): project cycle (from identification to completion)

6 Project Changes: processing project changes during project supervision

7 Development Policy Lending (DPL): project cycle (from identification to completion)

8 Corporate Reviews: procedures for regional operations committee/operations committee reviews

9 Contractual Remedies: application of contractual remedies

10 Legal – SIL: legal aspects of specific investment loans

11 Legal – DPL: legal aspects of development policy lending

12 Financial Management – SIL: financial management aspects of specific investment loans

13 Financial Management – DPL: financial management aspects of development policy lending

14 Procurement – SIL: procurement aspects of specific investment loans

15 Procurement Complaints: procurement processes relating to complaints

16 Procurement Non-compliance: procurement processes relating to non-compliance

17 Loan Management – SIL: loan management aspects of specific investment loans

18 Loan Management – DPL: loan management aspects of development policy lending

19 Loan Management – Application Review: processes relating to application review

20 Loan Management – Special Commitment or Application Problems: processes relating to special commitment or application problem

21 Loan Management – Amendments and Notices: processes relating to amendment/extension proc-essing

22 Loan Management – Refunds: processes relating to refund processing

23 Loan Management – Cancellations: processes relating to cancellation processing

24 Loan Management – Suspensions: processes relating to suspension processing

25 Loan Management – Loan Account Closing: processes relating to loan account closing - standard procedures

26 Loan Management – Loan Account Closing: processes relating to loan account closing - special pro-cedures

27 QAG: quality at entry and supervision

28 Safeguards – SIL: safeguard aspects of specific investment loans

29 Safeguards – Corporate Risk: safeguard procedures relating to corporate risk (QACU)

30 Debt Sustainability Analysis: relating to financial management

Attachment 1

114

* These two sub-processes were initially recorded as separate processes. After discussion with CFP man-

agement they were deemed to be sub-processes of the IDA allocation process and were consequently integrated into process number 1. We have not revised the numbering scheme. There are 28 processes that have been se-lected by Management for documentation.

Attachment 1

115

SAMPLE OF PROCESS FLOW DIAGRAMS Module 4: Country Assistance Strategy

Preparation

Final Review

Country Teamdrafts CAS /CASPR / ISN

InternalConsultations:SFR/FRM, LEG,Reg. Sectors

CorporateReview -

Upstream (Mod 8)

UpstreamRevisions to

CAS

Government /External

StakeholderConsultations

CorporateReview - Final

(Mod 8)

CountryConditions

Coordinationw ith IFC and

MIGA

CAS / CASPR /ISN Due

ISN / CASPR orCAS?

CASCR -External

Consultations

Updates toDraft CAS

Store UpdatedDraft in IRIS

WB MDReview

MD Clearance?

BoardDiscussion

BoardComments?

Disclosure

Store NextUpdated Draft

in IRIS

Consult w ithOPCS on Type

of Review s

Deliver CASCRto OED

Draft CASCR

Final Revisionsto CAS

RVP Review

RVP Approval?

OPCS Review sand Advises

CAS or ISN?

Projects inSAP

RM

CASPR?

Joint IFCproduct?

IFC Review

FRM/SFR /Legal

Consultation?

CoordinateUpdates w ithFRM/SFR and

LEG

SendDocumentation

to Board

OED Review ofCASCR

Store FinalCAS in IRIS

IncorporateComments

Consult w ithClient Gov. on

Changes

Client Issues?

YesNo

No

Yes

Yes

No

Yes

Yes

No

Yes

Yes

No

CAS

ISN / CASPR

CAS

ISN

Yes

No

1

2

3

Attachment 1

116

Module 8: Corporate Review

Preparation /Appraisal

CAS, PD, PADReady for Review

(Mods 4, 5 & 7)

ROC Review OC Review

PrepareDocumentation

Package

Require OCReview ?

Send ROCInvitations w ithDocumentationusing StandardDistribution Lists

ROC or OC?

OPCS Acts asSecratariat

OC Input fromSectors/Netw orks

ROC Input fromSectors /

Netow orks

Input fromOPCS

CirculateMinutes

UpdateDocumentation

CirculateDecision Note

CAS, SIL,DPL (Mods4, 5 & 7)

MD RequestPre-Negotiation

Review ?(Operations)

Pre-Neg.MD Review

AppraisalComplete

(Modules 5,7)

Pre-Neg. MDReview ?

CompletePre-Neg. MD

Review

Send OC Invitationsw ith

Documentationusing StandardDistribution Lists

Yes

No No

ROC OC

YesNo

Yes

1

2 2

117

Attachment 2: IAD Review of Management’s Assessment

Attachment 2

119

Attachment 2

121

Review of Management's Assessment of the Design Effectiveness of Internal Controls over IDA Operations and Compliance with its Charter and Policies

Background As part of its work program, Internal Auditing Department (IAD) has completed a review of “Management's assessment relating to the design effectiveness of key controls currently in place to en-sure compliance with the relevant Articles provisions and policies governing IDA’s operations”. This assessment comprises the first Part IA of a three-part assessment envisaged to satisfy Man-agement’s commitment “to carry out an independent comprehensive assessment of IDA's con-trol framework including internal controls over IDA operations and compliance with its charter and policies”. Management’s commitment was outlined in the IDA Fourteenth Replenishment report1, approved by the Executive Directors of IDA in March 2005, which identified a monitor-able action, targeted for CY05, the product of which was an ‘OED Assessment’. The balance of Management’s commitment will be met through Part IB: the assessment of the operating effectiveness of, or compliance with, the controls identified in Part IA; and Part II: the assessment of efficiency and effectiveness, including corporate governance and entity-level con-trols. Management indicated in its Revised Work Plan that its assessment would be conducted using the control framework issued by the Committee of Sponsoring Organizations of the Treadway Commission (COSO). Objective In accordance with our Terms of Reference, IAD’s objective was to review the basis of Man-agement's assessment and express an opinion on whether Management's assessment of the ef-fectiveness of internal controls over IDA operations is fairly stated based on the criteria estab-lished in Internal Control – Integrated Framework issued by COSO. Management’s assessment, originally outlined as Part A, was to “focus on the compliance portion of the assessment, namely whether the existing internal control framework over IDA’s operations pro-vides reasonable assurance to Senior Management and the Board that such operations are carried out in a

1 See “Report from the Executive Directors of the International Development Association to the Board of Governors, Additions to the IDA Resources: Fourteenth Replenishment, Working Together to Achieve the Millennium Development Goals” (approved by the Executive Directors of IDA on March 10, 2005).

Attachment 2

122

manner that complies with the provisions of the IDA charter and internal policies governing the fiduciary aspects of IDA’s lending operations”2. Part A has since been segregated into two parts (Part IA and Part IB) as outlined above. This segregation resulted from unsuccessful attempts to obtain documentation to test compliance and design effectiveness concurrently, as was originally in-tended by management. Scope IAD’s review examined Management's assessment relating to the design effectiveness of key controls currently in place to ensure compliance with the relevant Articles provisions and poli-cies governing IDA’s operations. Twenty-nine documented processes of the thirty processes3 deemed in-scope by Management were reviewed, including management’s methodologies for determining in-scope processes and for assessing the design effectiveness of internal controls. Management’s assessment and IAD’s review covered the period from July 1, 2005 to March 31, 2006. IAD did not review the following areas excluded by Management under Part IA:

• Overarching control framework for IDA including all aspects of corporate governance and entity level controls as well as efficiency and effectiveness of operations;

• Specific processes deemed out of scope by management for this assessment, namely

Economic Sector Work (ESW), Report on Observance of Standards and Codes (ROSC), Independent Evaluation Group (IEG) Processes, Internal Auditing Department (IAD) Processes, Country Policy & Institutional Assessment (CPIA), Post Conflict Performance Indicators (PCPI), Annual Report on Portfolio Performance (ARPP), the Inspection Panel, the Department of Institutional Integrity (INT), and the Results Assessment Framework of IDA;

• Process walkthroughs, from inception to completion, except for 3 processes (Country

Assistance Strategy, Investment Lending and Development Policy Lending);

• Operational Policies (OPs) and Bank Procedures (BPs) excluded by management, as out-lined in their report; and

• Compliance controls embedded in automated applications used in IDA operations4.

2 See “An Assessment of IDA’s Internal Control Framework: Revised Work Plan”, July 7, 2006, AC2006-0068 and “An Assessment of IDA’s Internal Control Framework – Proposed Work Plan, August 8, 2005, AC2005-0092, CODE2005-0078. 3 Management has not completed the documentation of the process relating to the “Debt Sustainability Framework”. We understand that this will be completed by management along with Part IB of the com-pliance assessment. 4 IT controls significant for financial reporting purposes were assumed to have been covered under man-agement’s Assessment of Internal Controls over Financial Reporting for FY06.

Attachment 2

123

Approach A highly compressed time schedule for the review was necessitated by management’s decision to segregate design effectiveness from compliance testing following the unsuccessful attempts by management to carry out this testing concurrently. As a result, IAD’s review, originally scheduled to follow completion of management’s assessment, was carried out simultaneously, with IAD conducting its own work independently and providing frequent and continuous feedback to management. As agreed with management, IAD applied all relevant concepts of Auditing Standard 2 (AS2)5 appropriately tailored for auditing operational compliance controls. IAD reviewed management’s scoping methodology and work plan. For the 29 documented processes (of the 30 in-scope IDA processes), IAD performed the following:

• Process Documentation: IAD reviewed high level process flowcharts, narratives of proc-

esses and control objectives, risks and key controls provided by management. IAD re-viewed whether key controls identified by management appeared adequate to satisfy con-trol objectives, and identified potentially missing key controls.

• Workshops/Review Sessions: IAD observed workshops/review sessions conducted with subject matter experts to validate process flow charts and narrative descriptions of individ-ual key controls provided by management, challenging, seeking clarification and identify-ing potential deficiencies as appropriate.

• Revised Process Documentation: IAD reviewed revised process descriptions incorporating

changes identified in the workshops/review sessions and potential control issues identified by management.

• Deficiency Tracker: IAD provided management with a list of 59 potential deficiencies in

documentation and/or design identified during our review, 55 of which were incremental to control issues identified by management. The following table outlines the nature of defi-ciencies by module.

5 Auditing Standard No. 2: An Audit of Internal Control over Financial Reporting Performed in Conjunc-tion with An Audit of Financial Statements (AS2) issued by the Public Company Accounting Oversight Board (PCAOB).

Attachment 2

124

IAD’s Review of Management’s Assessment of IDA Internal Controls: Part IA Potential Deficiencies Outstanding for Remediation – By Deficiency Type

MODULE Documentation Design Total Potential

Deficiencies DPL/SIL - 3 3 ROC/OC 1 1 2 FRM - ALLOCATION 5 1 6 LOAN MANAGEMENT 9 2 11 FINANCIAL MANAGEMENT 4 7 11 PROCUREMENT 15 7 22 SAFEGUARDS 1 1 2 QAG 2 - 2 Deficiencies 37 22 59

• Process Walkthroughs: IAD attended process walkthrough sessions convened by manage-

ment with operating personnel responsible for three processes (Country Assistance Strat-egy, Investment Lending and Development Policy Lending) to confirm the operation of process controls (for at least one transaction for each process) from inception to completion, as validated in the workshops/review sessions.

• Management's Report: IAD reviewed drafts of Management’s report and provided com-

ments as appropriate. General Observation This IDA assessment is the first comprehensive internal exercise undertaken by management to review an operational/compliance internal control framework. Furthermore, it appears to be unique in the multilateral development banking environment, and to our knowledge, in the broader international financial institution community. The effort underlying the commitment, the magnitude of which was clearly underestimated at the outset, should present substantial commensurate benefits: its results should provide a compelling baseline to identify opportuni-ties for streamlining IDA’s (and concurrently IBRD’s) operations and internal controls while significantly improving consistency and efficiency. Key Issues The following key issues were identified during our review of Management’s Part IA assess-ment: 1. IDA Processes Selected: Management’s methodology does not adequately rationalize its in-

tention to limit the scope of the review, as outlined in the IDA Control Review Methodology Note dated July 6, 2006, to determining compliance with fiduciary aspects of lending opera-tions in IDA’s charter, as the basis for deciding which IDA processes were relevant to the assessment.

Attachment 2

125

Management further limited its scope to processes applicable to Specific Investment Lend-ing (SILs) and Development Policy Lending (DPLs) as the two lending instruments repre-senting most of the value and volume, and which were deemed to be a representative proxy for other lending operations. Certain other IDA products, trust funds and grants that do not utilize IDA resources, and other miscellaneous operations outlined in the Articles were also excluded. We are not aware of any intended scope limitations of the commitment for a comprehensive assessment. By scoping out certain of IDA’s operations, a selective rather than a comprehensive “bot-tom-up” approach has resulted. Rationalization appears necessary to reconcile manage-ment’s interpreted scope with that described in the original commitment.

2. Information Technology (IT) Controls: Work has not yet been undertaken to identify sig-

nificant compliance controls embedded in automated applications. The separate Assessment of Internal Controls over Financial Reporting (ICFR) was neither designed nor intended to cover operations compliance controls, automated or manual. To include the assessment of automated compliance controls embedded in applications along with General Computer Controls (GCC’s) as part of the entity-level controls review in Part II will require re-work in the areas of documentation, confirmation, and testing to assess design and operating effec-tiveness of process/transaction level controls.

In order to reliably conclude its Part I assessment of process/transaction level controls, management will necessarily be required to carry out its assessment of key IT controls to-gether with Part IB.

3. Fraud and Corruption Controls: Identification and documentation have not been under-taken of fiduciary controls focused specifically on mitigating risks associated with fraud and corruption. Furthermore, an assessment of the adequacy of other identified controls to satisfy these specific objectives has not been carried out in Part IA. This would include con-trols to prevent and detect fraud and corruption, as well as ensuring that control implica-tions identified during fraud and corruption investigations are adequately addressed.

Management should specifically identify and assess key controls to prevent and detect fraud and corruption at the process/transaction level.

4. Outdated Operational Policies (OPs) and Bank Procedures (BPs): One objective of the as-

sessment by management related to the design effectiveness of the key controls that ensure compliance with policies and procedures governing IDA’s operations. Management has ac-knowledged in its report that “OPs and BPs … are not keeping pace with the changes on the ground that are being introduced from time to time”.

In principle, absent processes to ensure that policies are current, controls to ensure compli-ance with such policies would not be meaningful, even if current practices meet business needs.

Attachment 2

126

Management has committed to review current processes to facilitate more efficient and timely updating of OPs and BPs. Management has also committed to review the appropri-ateness of regional variances in their implementation.

5. Categorization and Remediation of Deficiencies: Analysis has not yet been carried out un-der Part IA of management’s assessment to determine whether identified deficiencies pose, in the aggregate if not individually, significant or material risks to the attainment of the con-trol objectives to which they relate.

In our view, in the absence of such determination, management has no sound basis upon which to conclude whether controls are effectively designed. Accordingly, it will be important for management to evaluate the significance of identified deficiencies prior to concluding on Part IB.

6. Document Retention and Accessibility: Management acknowledged significant difficulties

in obtaining timely access to relevant documents for compliance testing, which led to the segregation of assessments of control design effectiveness from operating effectiveness. In our view and experience, the inability to verify compliance with key controls represents a significant control design deficiency.

Management has committed to address the document retention and accessibility issue im-mediately.

7. Assessment of Entity-level Controls: Management has indicated that the review of entity-level controls, including tone at the top, the assignment of authority and responsibility, ap-propriate policies and procedures, and company-wide programs, will be carried out under Part II of the assessment.

The effectiveness of entity-level controls should have been as-

sessed prior to undertaking any assessment of controls at the process or transaction level, since controls at the organizational level often have a pervasive impact on controls at the process, transaction or application level. Therefore, any management conclusions on con-trol effectiveness as a result of Part I assessments will need to be reconsidered once entity-level controls have been examined.

8. Walkthroughs of Process Documentation: Management’s assessment of design effectiveness of internal controls under Part IA of the review included walkthroughs of process docu-mentation for 3 of the 29 documented in-scope processes (CASs, SILs and DPLs) from incep-tion to completion. The workshops/review sessions for the other 26 documented in-scope processes do not meet the criteria for walkthroughs outlined in AS2.

End-to-end process walkthroughs of all in-scope processes should have been conducted prior to management concluding on design effectiveness of internal controls, but in any case will need to be carried out by management prior to (or concurrently with) compliance test-ing in Part IB to support any conclusion.

Attachment 2

127

Overall Conclusion IAD committed in our original Terms of Reference to express an opinion at the end of Part A of the review (now Part IB) on whether management's assessment of the effectiveness of internal controls over IDA operations relating to compliance with IDA's charter and applicable policies is fairly stated. Any opinion delivered following Part IB would be subject to the outcome of the assessment of corporate governance and entity level controls in Part II. However, IAD strongly recommends reconsidering the relative cost-benefit of continuing im-mediately with the remaining phases of the assessment versus continuing after addressing sig-nificant deficiencies identified in Part IA.

___________________

___________________

Packiaraj Murugan Auditor-in-Charge

Thomas Ho Quen Hum Audit Manager