the world bank€¦ · · 2016-07-13the world bank report no:icr0000135 ... original target was...

Document of The World Bank

Report No:ICR0000135

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-44680)

ON A

LOAN

IN THE AMOUNT OF US$ 30.0 MILLION

TO THE

RUSSIAN FEDERATION

FOR A

DEVELOPMENT OF THE STATE STATISTICAL SYSTEM PROJECT

December 21, 2007

Poverty Reduction and Economic Management Russia Country Management Unit Europe and Central Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS (Exchange Rate Effective April 30, 2007)

Currency Unit = Ruble (R) R 1.00 = US$ 0.04

US$ 1.00 = R 25.72 FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS BEA Bureau of Economic Analysis Foundation M & E Monitoring and Evaluation BoP Balance of Payments MESI Moscow Institute for Economy and Statistics CANCEIS Canadian Census Editing and Imputation

System MOEDT Ministry of Economic Development and

Trade CBIS Computer Based Information System MOE Ministry of Economy (MOEDT’s

predecessor) CBR Central Bank of Russia MOF Ministry of Finance CIF Cost, Insurance and Freight (prices) NOE Non-Observed Economy CIS Commonwealth of Independent States OECD Organization for Economic Co-operation and

Development COFOG Classification of Functions of

Government OKVED All-Russian Classification of Kinds of

Economic Activity CPI Consumer Price Index O & M Operations and Maintenance CSDB Central Statistical Data Bank PAD Project Appraisal Document EC Economic Census PAR Public Administrative Reform EU European Union PDO Project Development Objectives FOB Freight on Board (prices) PetroStat Branch of RosStat in St.Petersburg FTP Federal Targeted Program PHRD

grant The Japan Policy and Human Resources Development grant

GDP Gross Domestic Product PIU Project Implementation Unit GFS Government Financial Statistics RF Russian Federation GFS 2001 Government Financial Statistics (2001) RosStat Federal State Statistical Service HBS Household Budget Survey SCC State Customs Committee HR Human Resources SDDS Special Data Dissemination Standard HRMIS Human Resource Management

Information System SNA System of National Accounts

IBRD International Bank for Reconstruction and Development

SNA93 United Nations’ System of National Accounts agreed in 1993

ICC Inter-Agency Coordinating Committee SPSS Statistical Package for the Social Sciences ICR Implementation Completion and Results

Report SSS State Statistical Service

ICT Information and Communication Technologies

STASYS Development of State Statistical System Project

ILO International Labor Organization StatCap Statistical Capacity Building Program IMF International Monetary Fund TSSAs Territorial State Statistics Authorities KPIs Key Performance Indicators UN FAO Food and Agriculture Organization of the

United Nations LAN Local Area Network WTO World Tourism Organization LFS Labor Force Statistics ZAGs Civil Status Registration offices

Vice President: Shigeo Katsu

Country Director: Klaus Rohland Sector Manager: Asad Alam

Project Team Leader: Misha V. Belkindas ICR Team Leader: Camille Nuamah

Russian Federation Development of State Statistical System Project

CONTENTS

Data Sheet

A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design ........................................................... 1 2. Key Factors Affecting Implementation and Outcomes........................................................... 4 3. Assessment of Outcomes ........................................................................................................ 8 4. Assessment of Risk to Development Outcome..................................................................... 18 5. Assessment of Bank and Borrower Performance.................................................................. 19 6. Lessons Learned.................................................................................................................... 22 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ....................... 23 Annex 1. Project Costs and Financing ...................................................................................... 24 Annex 2. Outputs by Component.............................................................................................. 25 Annex 3. Bank Lending and Implementation Support/Supervision Processes ......................... 38 Annex 4. Beneficiary Survey Results ....................................................................................... 40 Annex 5. Stakeholder Workshop Report and Results ............................................................... 42 Annex 6. Summary of Borrower's ICR and/or Comments on Draft ICR ................................. 44 Annex 7. Comments of Cofinanciers and Other Partners/Stakeholders ................................... 61 Annex 8. List of Supporting Documents .................................................................................. 62

MAP

A. Basic Information Country: Russian Federation Project Name:

Development of State Statistical System Project

Project ID: P050487 L/C/TF Number(s): IBRD-44680 ICR Date: 10/26/2007 ICR Type: Intensive Learning ICR

Lending Instrument: SIL Borrower:

RUSSIAN FEDERATION (MINISTRY OF FINANCE)

Original Total Commitment:

USD 30.0M Disbursed Amount: USD 29.9M

Environmental Category: C Implementing Agencies: Bureau of Economic Analysis Foundation Federal State Statistical Service Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 02/04/1998 Effectiveness: 05/31/2000 05/31/2000 Appraisal: 11/10/1998 Restructuring(s): 12/23/2004 Approval: 05/13/1999 Mid-term Review: 05/27/2002 Closing: 06/30/2004 12/31/2006 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Low Bank Performance: Satisfactory Borrower Performance: Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Satisfactory Government: Satisfactory

Quality of Supervision: Satisfactory Implementing Agency/Agencies: Satisfactory

Overall Bank Performance: Satisfactory Overall Borrower

Performance: Satisfactory

C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance Indicators QAG Assessments

(if any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

No Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Satisfactory

D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) Central government administration 100 100

Theme Code (Primary/Secondary) Analysis of economic growth Secondary Primary Economic statistics, modeling and forecasting Primary Primary Infrastructure services for private sector development Primary Not Applicable Legal institutions for a market economy Secondary Secondary Macroeconomic management Secondary Secondary E. Bank Staff

Positions At ICR At Approval Vice President: Shigeo Katsu Johannes F. Linn Country Director: Klaus Rohland Michael F. Carter Sector Manager: Asad Alam Robert J. Anderson Project Team Leader: Misha V. Belkindas Misha V. Belkindas ICR Team Leader: Camille Nuamah ICR Primary Author: Camille Nuamah F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The development objectives of the Russia Development of the State Statistical System Project (STASYS) were to support an improvement in macroeconomic management by providing a required information base and to improve the availability of data need for a market economy. This was to be achieved by focusing on (i) eliminating data gaps in the macroeconomic

framework used for economic policy and management; (ii) capturing new and dynamic economic and social activities; (iii) meeting the needs of analysis of policy makers and the business community; (iv) introducing international standards and methodologies; (v) increasing the efficiency and productivity of statistical production through improved organization, data collection and processing methods and equipment; and (vi) improving the quality of analysis and the monitoring of economic development along with the modernization of decision-making methods for economic policy. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target

Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: Key statistical products improved for macroeconomic planning

Value quantitative or Qualitative)

Key national accounts, including balance of payments, not sufficiently adjusted for informal activity

None None

Surveys conducted, methodologies improved and estimates for informal activity introduced in national accounts

Date achieved 05/31/1999 12/31/2006 Comments (incl. % achievement)

Indicator 2: Data collection and processing systems improved


Rudimentary computer systems did not allow for integrated processing and analysis of data across RosStat system

New systems in place in more than 50% of the regions

None

An integrated processing system with 17 subcomponents developed and successfully tested in 15 pilot regions


Original target was over-ambitious, but feasible target were fully achieved. The full rollout is underway in the Federal Targeted Program on Statistics and is supported by the follow-on project, STASYS2

Indicator 3 : Staff skills upgraded


Sporadic training program and few facilities in place.

More than 24% of staff trained and retrained

Over 5500 (24%) RosStat staff with improved skills and knowledge in statistical methodologies


This outcome has been accompanied by the development and implementation of an up-to-date and sustainable training plan, improved facilities and increase budgetary resources.

(b) Intermediate Outcome Indicator(s)



approval documents)


Values



Indicator 1 : Transition from census-based to survey-based statistical estimation

Value (quantitative or Qualitative)

Majority of economic statistical products based on census-based data collection

Infrastructure for survey implementation is created

None

Sampling frames developed, data collection re-organized, new methods tested and piloted including for automatic control and editing of census and survey data

Date achieved 05/31/1999 12/28/2006 12/31/2006

Comments (incl. % achievement)

Although few products have actually been converted to surveys, excellent methodological and technical preparation has been made for their full-scale conversion. While baseline Population and Agricultural Censuses have been conducted with improved methods, Economic Census is delayed.

Indicator 2 : The SNA structural indicators revised to harmonize with the international standards


RF using outdated country specific classifiers based on command economy

The RosStat approved the recommendation on SNA structural indicators

The All-Russia Classifier of Economic Activities is developed and approved.


Indicator 3 : Monitoring of the Performance Based Budgeting


Performance-based budgeting not yet in place or approved.

System of statistical indicators for the PBB is designed and launched.

Concept for the organization of the a system of PBB indicators is completed

Date achieved 05/31/1999 12/28/2006 12/31/2006 Comments (incl. % achievement)

This objective was introduced in 2005 when the RF approved the move to performance based budgeting, and the role of RosStat has been largely accomplished.

Indicator 4 : New Law on Statistics in place


Draft Law on Statistics not yet prepared

Draft Law is submitted to the Parliament (State Duma)

Law on statistics was signed on November 29, 2007


The objective was not achieved during project implementation, but subsequently achieved almost one year after project closing

Indicator 5 : Development of data base facilitating the integration of the statistical and administrative information resources.



approval documents)


Values




No data base in place The data base is in place The data base is in

place and operational


G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 06/30/1999 Satisfactory Satisfactory 0.00 2 12/28/1999 Satisfactory Satisfactory 0.00 3 03/03/2000 Satisfactory Satisfactory 0.00 4 06/30/2000 Satisfactory Satisfactory 0.30 5 12/26/2000 Satisfactory Satisfactory 0.80 6 06/27/2001 Satisfactory Satisfactory 0.98 7 11/12/2001 Satisfactory Satisfactory 1.17 8 12/27/2001 Satisfactory Satisfactory 1.61 9 06/27/2002 Satisfactory Satisfactory 2.77

10 12/24/2002 Satisfactory Satisfactory 6.39 11 06/26/2003 Satisfactory Satisfactory 8.67 12 12/29/2003 Satisfactory Satisfactory 14.25 13 06/08/2004 Satisfactory Satisfactory 20.10 14 12/27/2004 Satisfactory Satisfactory 23.39 15 06/30/2005 Satisfactory Satisfactory 25.99 16 12/15/2005 Satisfactory Satisfactory 26.74

H. Restructuring (if any)

ISR Ratings at RestructuringRestructuring

Date(s)

Board Approved

PDO Change DO IP

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

12/23/2004 N S S 23.03

Additional counterpart financing to extend IT pilots and add two new activities. Required changes to disbursement schedule, extension of closing date and

ISR Ratings at RestructuringRestructuring

Date(s)

Board Approved

PDO Change DO IP

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

adjustment to performance indicators.

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal This Development of the State Statistical System Project (STASYS) was prepared during a time when the Russian Federation (RF) was undergoing both political and economic turmoil following the profound collapse of the economy at the start of transition. Nevertheless, a reformist and technocratic government had taken office in the spring of 1998 and it was widely accepted that there was a need for accelerating structural reforms as the basis for renewed economic growth. The financial crisis of mid-late 1998 further confirmed that a comprehensive and accelerated program of economic reforms would be required. The authorities clearly recognized the need to improve the quality of statistical information as a key to strengthening macroeconomic management. The transition to a market economy posed major challenges to the statistical system related, inter alia, to the emergence of a private sector, expansion of economic activities especially in services, and the changing role of the state in economic management. These changes would eventually require fundamental changes in the approaches to data collection and compilation issues. In addition, Goskomstat – the State Statistical Committee – had suffered from a dramatic decrease in the funding for statistical services both in relative and real terms. Year after year the funds earmarked for statistical programs were not disbursed as planned with capital expenditure being the most affected. The share of Goskomstat (including its training institutions) in the total Federal budget had fallen by about 30 percent between 1995 and1997. Nevertheless, by the time of appraisal however, the RF had met some of the challenges systematically and successfully, with significant assistance from development partners. In the area of macroeconomic statistics, the statistical agency had begun its preparation of new macroeconomic indicators, in compliance with the 1993 System of National Accounts (SNA), started collecting data on prices and introduced into statistical practice the Consumer Price Index (CPI) and other price indexes. The Bank was already supporting: (i) strengthening the estimation of national accounts, input-output analysis and household and labor force surveys by Goskomstat under the Bureau of Economic Analysis Project; and (ii) surveys to develop a set of poverty indicators under the Social Protection Investment Loan. In addition, the compilation of Balance of Payments (BoP), money and banking, government finance statistics and new types of social statistics had been introduced and data were being collected and compiled. Key problems remained including the fact that balance sheets of all business entities and financial institutions were still done using Soviet-type accounting standards and many new enterprises did not compile data according to statistical reporting requirements. As such, the supply of good quality data and standards for macroeconomic compilations and analysis was limited. Certain economic activities carried out by the shadow economy either failed to be captured or were covered only partially by official statistical reports. This placed additional challenges on statisticians to make ad hoc adjustments to primary information. In 1997, the authorities approved a Federal Earmarked Program for the Development of the Statistical System of the Russian Federation for 1997-2000. The World Bank responded by supporting the refinement of the development strategy. It was soon recognized that, for the Goskomstat to keep apace with data needs for sound macroeconomic management and to facilitate private investment, a radical overhaul of the system of data collection and processing, as well as the introduction of modern estimation and analytical techniques would be needed. Key

2

modifications to the authorities’ plans included: (i) broadening the scope of reform of the wider statistical system, i.e. including other government agencies, besides Goskomstat, involved in the production of statistics and emphasizing the need to improve coordination among them; (ii) a focus on building management capacity for the statistical system; (iii) addressing the legal framework; (iv) a more comprehensive and deeper reform of data collection and processing methods that was originally envisaged; and (v) added emphasis on improving dissemination and building relationships with end-users.

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The development objectives of STASYS were to support an improvement in macroeconomic management by providing a required information base and to improve the availability of data needed for a market economy. This was to be achieved by focusing on: (i) eliminating data gaps in the macroeconomic framework used for economic policy and management; (ii) capturing new and dynamic economic and social activities; (iii) meeting the needs of analysis of policy makers and the business community; (iv) introducing international standards and methodologies; (v) increasing the efficiency and productivity of statistical production through improved organization, data collection and processing methods and equipment; and (vi) improving the quality of analysis and monitoring of economic development along with the modernization of decision-making methods for economic policy.

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and Reasons/Justification

The development objectives were not revised during implementation.

1.4 Main Beneficiaries The main beneficiaries as identified by the PAD were (i) government agencies and the Central Bank involved in economic policy making; (ii) domestic and foreign investors, producers and traders which needed good statistical information for informing business decisions, and (iii) international organizations, foreign governments and multi-national corporations which require standardized economic and social data on the Russian Federation for comparative purposes. The project would also have benefited respondents, both domestic and resident foreign individuals and entities, by reducing the burden and cost of statistical reporting and surveys.

1.5 Original Components (as approved)

The project included ten original components divided into three main areas: (A) institutional strengthening of Goskomstat; (B) institutional strengthening and technical assistance for data collection in other agencies; and (C) project management.

The first area was to support strengthening the management, technical skills, infrastructure and data collection, processing and dissemination processes at the State Statistical Bureau. The first Component aimed to improve the organization and management of Goskomstat itself including its legal arrangements, inter-agency coordination, dissemination and relationship with users; and the management of what was expected to be a long-term reform process. The second Component supported a large training program to help the Goskomstat staff update their knowledge of

3

modern statistical concepts and methodologies and to acquire new technical competencies in areas specific to the production of good quality statistics for the Russian Federation. It also sought to strengthen the management, design and facilities for Goskomstat’s ongoing training programs. The third Component focused on upgrading statistical infrastructure, mainly focusing on survey design and classifications systems, as well as managerial and planning facilities, and reforming the framework for stakeholder relationships. The fourth and largest Component supported a large program to reorganize and modernize data collection, processing and dissemination through the preparation of a large scale program to replace economic/business censuses with sample surveys and the acquisition of new hardware and software system for survey processing, data analysis and dissemination of statistics.

The second area of the project targeted other data collection agencies with a variety of program to improve specific statistical products. Support was to be provided to: (i) the Ministry of Finance and Ministry of Economy for improving government financial statistics and compilation of the general government sector in the national accounts (Component 5); (ii) the Central Bank for the expanding the coverage of financial statistics to non-bank intermediaries and auxiliaries; (Component 6) and together with Customs Committee for improving the quality and coverage of balance of payments statistics (Component 7); the Ministry of Economy again to strengthen its capacity to provide short and medium term economic forecasts (Component 8); and to the Registry Department to acquire computer facilities and software for inputting the initial data as a basis for Goskomstat to begin electronic processing of vital statistics (Component 9.)

The final area of the project (Component 10) supported the establishment of a directorate to manage the project and the development of the state statistical system.

1.6 Revised Components

During implementation, a number of changes were made to the components. First, the Bank and the Government agreed to subsume some activities for strengthening government financial statistics originally planned under the Bureau of Economic Analysis (BEA) project, under Component 5 of this project.

Second, Component 6 – Introduction and Compilation of Financial Statistics was dropped because of lack of interest on the part of the Banking Supervision Department at the Central Bank of Russia (CBR) and Component 9 – Improvement of Vital Statistics - was substantially reduced also in part due to lack of engagement by the Registry Department. The latter was replaced by technical assistance for the implementation of the All-Russia Population Census in 2003.

Key activities and performance indicators under components 3 and 4 were reorganized to consolidate activity on upgrading survey methodologies and facilities under subcomponent 4.1 on economic statistics which focused mainly on replacing censuses with sample surveys and under subcomponent 4.2 on the household sample survey system. As a result, at the mid-term review, Component 3 was substantially scaled back to comprise only the development of regulatory and legal documents and the introduction of new methodological approaches for compilation. Managerial and planning facilities were to some extent covered by Component 1. The development of a classifications unit was eliminated and the development of an all-Russia product classifier by economic activities was included under Component 8.

A new Component (11) was added to separate the work on the Federal Customs Service database from the rest of the work on the Balance of Payments statistics. No amendments to the loan agreement were required by these changes since they fit within the general description of project

4

objectives as included in the legal documentation. These actions were also consistent with the applicable Bank policies and practices at the time.

1.7 Other Significant Changes (in design, scope and scale, implementation arrangements and schedule, and funding allocations) The key changes during implementation include the restructuring of a number of components (discussed above) and a significant increase in counterpart financing from the Government to permit additional activities within the original project objectives. The latter resulted in an increase in total project size from $39 million to $55 million. This was administratively implemented by a significant reduction in the share of Bank financing for specific disbursement categories in the loan agreement. Notably, this was the first Bank project in Russia to accommodate a new co-financing scheme that maintained both financial and technical involvement of the Bank in an important reform area, despite increases in government’s own resources. These changes also required a second extension of the closing date by some 12 months to allow for full implementation of the scaled up technical solutions. The loan agreement was amended in October 2002 to allow project proceeds to finance local transportation and insurance of goods within the Russian Federation, and to change some of the procurement thresholds1. The closing date for the project was first extended from June 30, 2004 to April 30, 2005 to allow for completion of original project activities including the agricultural census methodology, the new classification of products by economic activity and inputs into the new chart of accounts in public administration. In December 2004, the Government requested the Bank to extend project activities and proposed additional counterpart financing by reducing the Bank’s co-financing share from 80 percent to 20 percent. These extensions and additions included further replication of technical solutions utilized in the information system to a few additional regions, and the development of statistical indicators for the RF’s ongoing effort to institute performance-based budgeting. The loan agreement was then amended to allow for increasing the Russian share of financing to 80 percent, to extend the closing date by some 12 months to December 2006 and to include four new performance indicators (see Annex 2). At the end, Government counterpart financing had been increased by some US$24 million over the original estimates (see Annex 1).

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry (including whether lessons of earlier operations were taken into account, risks and their mitigations identified, and adequacy of participatory processes, as applicable)

The project was based on very sound background analysis and preparation. Preparation and quality at entry benefited from a thoroughly prepared, with Bank assistance, development strategy that was backed up by a Federal Targeted Program of financing for strengthening the state statistical system. Work on the development strategy was also supported by a PHRD grant for preparation financed by the Japanese government. The preconditions of strong government ownership were also demonstrated by ongoing efforts to improve key statistical products, and

1 These changes included increasing the limits on International Shopping to $40K and National Shopping to $50K (overall $600k) the aggregate limit on SBQS to $702K.

5

were supplemented by Bank experience under two ongoing projects. The ultimate design of the project, while clearly addressing key activities and outcomes, was also flexible enough to allow for the Government and the Bank to take advantage of opportunities that arose during implementation, including the population and agricultural censuses, and the poverty assessment work, to extend the outcomes to other areas of data production and statistical analysis. However, there were a number of key issues in project preparation and design that ultimately affected implementation.

The development objectives to “support an improvement in macroeconomic management…” and while appropriate for what was understood to be a long-term statistical reforming process, may have been a bit too broad for an individual operation. STASYS project focused on mainly strengthening the system for producing good quality statistics along very specific lines – management capacity, statistical infrastructure, staff knowledge and systems modernization – and improving the quality of selected statistical products. However, the overly broad focus ultimately did not materially affect project outcomes. The objectives were, and still remain, relevant to the overall development priorities for the Russian Federation.

The choice to use an external project implementation unit (PIU), the BEA, was extensively debated within Government. While this debate and the final choice did cause delays at the outset, because of the need to legally restructure the BEA, which was rather effective in keeping the administrative side of project implementation running smoothly. It is not clear whether Goskomstat, itself being subject to organizational restructuring, would have been able to maintain continuity in these administrative matters while engaging in the planning and implementation of project activities and absorbing its own share of technical assistance. The BEA was able from the very beginning to, rather flexibly, strengthen its own staff capacity with the help of PHRD grant resources, to improve financial and implementation reporting, and to oversee the complex procurement processes that emerged. However, a by-product of this arrangement was that the BEA as an external body may not have had sufficient influence to support improved coordination on project issues among the various data collecting agencies, when the Inter-Agency Coordination Council performed this function less efficient as expected. The implementation period of four years was too short for, basically, a greenfield project of this breadth. First, despite using the BEA which had experience in implementing World Bank projects, there were significant delays in achieving effectiveness, which eventually took almost one year. Second, the required administrative procedures of the RF government, for example, have proven to be a common barrier to the timely preparation and start-up of Bank projects. This was no less the case for the STASYS project. Third, it is not clear that the full complexity of the systems modernization planned under Component 4 was understood at the time of appraisal. Finally, the level of ownership among the various agencies involved in the project was uneven, resulting in a number of components being dropped or reduced. However, it is not clear whether additional consultations during preparation would have revealed these problems, upfront. The project design ultimately permitted the flexibility to add support for improving other data products when the opportunities did emerge. This lesson has been fully internalized in the follow on project – STASYS 2, which was approved by the Board on June 28, 2007.

6

2.2 Implementation (including any project changes/restructuring, mid-term review, Project at Risk status, and actions taken, as applicable) Aside from the restructuring described above in Sections 1.6 and 1.7, a number of systemic developments in the Russian public sector during the project life did significantly affect progress on project implementation. The first were the serious fiscal constraints which followed the Russian financial crisis for some years and led to significant delays in receiving counterpart financing during the early years of implementation.

The second systemic issue was a long wave of state-wide public administrative reform (PAR) in the RF. While it did benefit the State Statistical Committee in critical ways, it may also have reduced the commitment to, and possibly the Committee’s flexibility to act on, the internal organizational restructuring which was an essential part of the project’s development strategy. At the start of the project, Goskomstat was downgraded by the Government of Russia, then restored to the level of “Committee”. In addition, the state-wide PAR program also mandated a series of internal organizational changes to the Committee over the life of the project, in part leading to a slowdown on decisions and activities by RosStat itself on the same issue. Even after, RosStat officials have suggested that they were awaiting further instructions from the broader PAR effort as an input into organizational restructuring. Towards the end of the project, the PAR reform also shifted the focus of development of the overall statistical system away from strengthening the quality of essential macro-aggregates toward development of local indicators for decentralization and program budgeting.

A third systemic issue involved RF procurement restrictions with respect to the use of international consultants which severely limited the pool for the needed, high quality technical assistance and also caused a number of delays in implementing key project activities.

Finally, there was a near-suspension of the Bank’s portfolio in the RF for almost one year in late 2003 through 2004 as the authorities undertook a review of implementation arrangements, which impacted the pace of project implementation during that time.

A number of project-specific problems also affected implementation. These included a long learning curve (already discussed in Section 2.1), insufficient coordination leading to the limited participation by some of the other data collection agencies, and the delays in the passage of the new Law on Statistics.

The Law on Statistics was submitted to Parliament in September 2007 after project closing. These delays contributed to RosStat’s reluctance to assume full leadership and convening power across the numerous agencies in the statistical system, but may be resolved now with the imminent passage. It is not clear how much buy-in from these other agencies there was during preparation. And during implementation, the project’s Inter-Agency Coordination Committee (ICC) was not strong enough to effectively resolve the numerous coordination issues, which resulted in a few of the smaller components falling by the wayside. That said, there was enough flexibility in the structure of the project and the responses of the Bank and government teams to grasp some opportunities for “filling data gaps and expanding coverage of economic activities” where they emerged. Several key examples are the addition of the support to the Customs Department, pilot testing of the Agricultural Census, and support to the Population Census.

Relatedly, analysis of the procurement plan reveals that there was an important amount of refocus and reallocation away from activities geared toward system-wide institutional development and

7

organizational restructuring toward IT and methodological issues. Some of this may also have been in response to a long wave of state-wide public administrative reform in the RF to which the Goskomstat/RosStat was subject, in effect superseding their own efforts. Even so, the overall set of financed activities comprise a coherent program that has achieved the majority of the original Component objectives, as well as supporting additional development in some needed areas, as they arose.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization While the STASYS project preceded the current effort in the Bank to rely on quantifiable project outcome indicators, the current Results Framework methodology was part of project design. A large number of, mainly, process-oriented, performance indicators were specified in the Project Appraisal Document which made earlier reporting on the “results” somewhat cumbersome and subsequently, quite limited in its effect on project implementation.2 Starting with the mid 2005 supervision report, the team did refocus the indicators on a few substantive issues that had emerged as some of the key priorities during implementation. These were the improvement of the data collection and processing systems, upgrading of staff skills, methodological improvements to the national accounts data, integration of statistical and administrative data, and passage of the Law on Statistics. A more focused results framework in the new PAD for STASYS2 would be useful to more effectively shepherd project outcomes. That said, over-specification of what is a very complex development agenda, and in the face of strong ownership, would not be appropriate. The mission understands that RosStat has developed a set of modern performance indicators for the FTP, which should be appropriately mined for this purpose. Also the IMF will begin to publish SDDS report cards from next year, which might be useful as one monitoring indicator.

2.4 Safeguard and Fiduciary Compliance

The project did not involve any environmental nor other safeguard issues, and was rated accordingly. Of the 23 audit and financial reports received during the life of the project, four were received late, and six had issues which were raised upon review and resolved. In the initial years there were some minor delays with the submission of financial reports, but these were soon resolved. The PIU successfully implemented project/financial monitoring report-based disbursements during the life of the project as originally intended. No major procurement compliance issues were raised during implementation, notwithstanding the various amendments which were needed to resolve market related issues for consulting services, discussed below.

A few fiduciary issues required resolution during the project. The first and most major involved the hiring of government agencies for local consulting contracts. The main rationale, discussed at length between the Bank and the Government, was the retention of a good number of RF’s technically skilled personnel in the field of statistics in state-owned agencies, mainly educational institutions. After a few competitions for local consulting services among the private sector failed to yield appropriately qualified firms, it was decided to allow government bodies to participate in the bids on a limited basis, to be reviewed in each instance. The second related to the prohibition from using loan proceeds to finance local transportation of goods which became

2 The Loan Agreement did include a more limited set, although still process-oriented, indicators.

8

problematic with the procurement of large quantities of computer hardware primarily intended for Goskomstat’s local offices in the Russian hinterland.

2.5 Post-completion Operation/Next Phase A follow-on project which will continue the modernization process – STASYS 2 – was approved by the Bank’s Board of Executive Directors on June 28, 2007. This project will support the implementation of the third Federally Targeted Program for the development of state statistics in Russia and aims directly at increasing the availability of data for the Russian Federation, in particular macro-aggregates and Millennium Development Goal indicators conforming to international standards. It will also support the rollout of the technological solutions developed under the first project to the national system.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation (to current country and global priorities, and Bank assistance strategy) Despite numerous changes to the structure of components, mainly to iron out overlaps in some of the components and to channel the flexibility/breadth of the project design onto specific improvements, the project remained focused on its key objectives of (i) improving the coverage, quality and availability of statistical products for the market economy; and (ii) increasing the efficiency of the production of statistics in the Russian Federation. These objectives remain relevant even today, because of the long term nature of statistical development in a transition economy. The implementation arrangements of having an external PIU and a multi-agency Coordination Committee although it did not yield the intended efficiency (in terms of start up) early on, were largely supportive of the project’s achievements. The external PIU allowed RosStat some breathing room to formulate its methodological and training needs, to elaborate on the terms of references as well as providing a flexible avenue to engage the relevant technical skills needed to support the complex IT systems, while RosStat and other agencies were subject to the constraints of the state-wide administrative and civil service reform programs. As evidence of its relevance, this implementation arrangement will be continued under the follow-on project.

3.2 Achievement of Project Development Objectives STASYS had two primary development objectives: (i) to provide a required information base for an improvement in macroeconomic management; and (ii) to improve the availability of data needed for a market economy. Both these objectives were substantially achieved with a range of new and improved statistical data products, internationally comparability of data, better dissemination and the strengthening of economic modeling and forecasting activities using the new data. These objectives were to be achieved by efforts to: (i) Eliminate data gaps in the macroeconomic framework used for economic policy and management.

Notwithstanding this very ambitious objective, the project did address some key data needs in the macroeconomic framework mainly through improvements in methodology. The main statistics of national accounts, government financial data, balance of payments and, notably, inflation were all

9

being measured in the Russian Federation at the time of project preparation. However, concerns about the quality of various issues led to the duplication of estimation between RosStat and various administrative departments in the Government, with the result that there were often duplicate and different publications of core statistical data. This situation made it difficult for policy makers, private businesses, independent think tanks, media and academics to engage in both robust planning and meaningful dialogue about macroeconomic issues. In addition, key estimates, in particular, for capturing new forms of economic activity as well as the informal and underground economy, were missing from these accounts.

As seen from sections (ii) and (iii) below, the STASYS project supported substantial improvements in the methodological approaches in all of the areas mentioned above, and added a large number of new statistical products, mainly relating to labor market. As a result, there has been a marked reduction of duplicate publications of data coming from the public sector. Towards the end, the project also supported the government’s efforts to institute performance budgeting by preparing a concept for the compilation of administrative date for this purpose.

Only in one planned area, that of debt statistics did the project fail to contribute to a measurable improvement. There were no activities implemented under the project for this subcomponent. According to MOF, there remain outstanding legal issues regarding the measurement and presentation of debt related to whether it is measured by currency (MOF) or residency (BoP) and an outstanding activity on rationalizing institutional arrangements.

(ii) Capture new and dynamic economic and social activities. The project was instrumental in providing a variety of opportunities for RosStat to address new data needs as they emerged along with the development of the market economy. A number of these were products of the effort to transit from an all-census data collection framework to an integrated census-sample survey framework described below. Through extensive technical assistance to the All Russia Population Census in 2002, All Russia Agricultural Census in 2006, and preparations for the All Russia Economic Census in 2008, the project helped to refine and expand various statistics on:

• the labor force, including the compilation of data on economic activities by type of household and analysis of the occupation/qualification structure of the labor force;

• individual entrepreneurs, a segment of the economy hitherto largely unmeasured and resulting in gross underestimation of a large component of informal economic activity; and

• agricultural production. Through efforts to upgrade the balance of payments statistics, the project helped produce new statistical output on the trade in goods, international trade in some services and informal trade through various surveys undertaken to refine the Balance of Payments; and the computerization and expansion of the Customs Service systems for data compilation, analysis and dissemination.3

3 However, a primary user – CBR, which is officially responsible for compiling BoP statistics – has more limited functionality than other users, because it is not a ‘budget-financed” institution and is therefore considered to be outside of government.

10

Others resulted from the (flexibility to make) opportunistic use of project resources to satisfy government demand for data to inform current economic policy questions and reflects the ongoing need to balance implementation of the “master plan” for development of the statistical system with the needs of its (primary) users. Some of the new statistical products and expanded coverage that were developed with project support include:

• an aggregate index of economic development at sub-regional level directed at the Russia’s ongoing efforts to address regional disparities and related efforts to extend reform of intergovernmental finances to lower levels of local government;

• an information and statistical database for the assessment of national economic potential for innovation, including types of innovative activities, sources of funding for innovation, and methods use to protect intellectual property. Statistical enhancements included amendments to data collection forms and compilation methods;

• a related system of indicators for measuring competitiveness of the Russian economy; and

• a concept for the organization of performance monitoring indicators for budgetary planning units to support the ongoing performance budgeting and public administrative reforms in Russia.

(iii) Meeting the needs of analysis of policy makers and the business community. The project was instrumental in improving the quality of macroeconomic, microeconomic and social data for use by policy makers and the business community, as illustrated by the enhancement to data products used by these groups. However, there were very few activities under the project aimed at reaching out to non-governmental stakeholders, other than expert users – researchers and academics – in a systematic fashion other than the upgrading of the website, open access to microdata from the household budget surveys used for the World Bank’s program of poverty analysis, and acquisition of a new GIS-based data presentation tools. This is evidenced by the RosStat’s very ad hoc and rudimentary implementation of a user survey basically only to governmental users of its output (see Annex 5).

Recognition is growing within RosStat on the need to strengthen interaction with non-government and non-expert users of its products, but it is does not appear yet to be a priority. Regarding expert users, there is a Methodology Commission with external representatives from universities and research centers. There are also some individual efforts to reach out to users in the design of new surveys and during the development of some of the new analytical tools. A few officials noted that the Law is also needed to regulate the relationship with users, but it is not yet clear whether the proposed new Law will achieve this adequately. There is no documentation that gives clear government policy regarding dissemination and access of official data produced within the State Statistical Service. RosStat and statistics units in line ministries and regional statistics offices generally supply data to users on request within some explicit procedural framework, but none of them has developed a data access policy.

From the user perspective, participants from the private sector, academia and media in a workshop organized for this report (see Annex 5) reported that, despite the persistent legacy of distrust in Russian officialdom, confidence in RosStat outputs by users is increasing. The relationship with users at the regional level in St. Petersburg is quite strong, as evidenced by the regular and deep interaction between PetroStat, local academia and local public administration (also see Annex 5 for more details). However, there remains a need for RosStat to strengthen

11

presentation of sources and publication of metadata, to publicize new analytical capabilities, to focus on public education on statistics in order to reinforce and accelerate this growing confidence, and to routinely investigate major anomalies with other data sources, particularly at the local level.

That said, RosStat is already disseminating a large number of reports in hard copies and is placing data on its website. Workshop participants admitted that there has been substantial improvement in the quantity of data disseminated by RosStat, especially on its website. In fact, the international Internet-ranking firm Rambler rates this site among the top four administrative sites in Russia. Data dissemination efforts have further been strengthened by RosStat’s subscription to the IMF’s Special Data Dissemination Standard (SDDS) although there are a few outstanding issues. This is a remarkable improvement. Notwithstanding the progress achieved, RosStat needs to make further improvements to overcome existing limitations in its current dissemination practices. Further upgrades to the RosStat website are needed, in particular to create more dynamic links to data and capacities for downloading in forms accessible to analytical software.

A second group of stakeholders, only tangentially considered under the project, are survey respondents. RosStat senior management is keenly aware of this issue and its impact on response rates and quality of information reported on surveys, with one commenting, “Ministries and agencies force us to torture respondents.” However, as yet there is no measurement of respondent burden, and no particular policy for dealing with respondents. The general impression of RosStat staff are that enterprises are becoming more resistant to providing data, resulting in both lower response rates and increasing concerns about the quality of data provided. Assuming gradually broader application of the integrated data collection system, it should be possible to measure response rates more systematically as an efficiency and accuracy indicator. For example, the pilot survey of individual entrepreneurs indicated that face-to-face interview was a much more effective way of collecting data than the self-completed mail-out, mail-back questionnaire or telephone interview. Likewise, measurements of respondent burden should be designed and implemented, at least for economic surveys.

(iv) Introducing international standards and methodologies. This objective was addressed primarily through two major inter-related efforts under Components 2, 3, 5 and 7 encompassing (a) enhancement of key statistical products – Russia’s system of national accounts (SNA), government financial statistics (GFS) and balance of payments (BoP), and a wide range of other products; (b) introduction of an internationally-compatible classification system; and (c) a massive effort to upgrade staffing skills. National accounts. Under the STASYS, RosStat has completed a number of activities related to the implementing the international guidelines as prescribed by the United Nations’ System of National Accounts agreed in 1993 (SNA93). A major review of the sources and methods compiling national accounts in Russia was undertaken with technical assistance from the project and the International Monetary Fund. Data collection and compilation methods were revised, re-examining existing and new data sources. Following this review, RosStat has utilized project financing to make a number of key enhancements, including:

• the introduction of commodity-flow methods for compiling detailed flows of goods and services;,

• improvement of computational techniques for obtaining GDP estimates based on the supply and use tables compiled through the commodity-flow method;

• classification of flows and stocks;

12

• development of a method of recording investment in non-financial assets considered appropriate for use in Russia’s system of national accounts;

• development of a method for compiling tourism statistics based on the recommendation of the World Tourism Organizations;

• development of experimental estimates using labor input approach and data from the labor force survey4;

• development of a methodology for concentration of production; and • compilation of balances of food resources.

The next steps should be to: (i) implement the existing program of surveys and data collection that have been identified, and move towards the implementation of the full sequence of accounts and balance sheets in the 1993 SNA; and (ii) develop appropriate indicators for the imputation and updating of various components of the national accounts aggregates. Government financial statistics. In 2005, the Ministry of Finance (MOF)5 completed a major review that involved building a new budgetary classification and moving to an accrual method of accounting at regional and federal levels. Beginning January 2006, the government finance statistics produced by MOF have been based on the methodology and guidelines given by the IMF’s Government Finance Statistics 2001 (GFS 2001) manual and the UN’s Classification of Functions of Government (COFOG). The statistics will be published on quarterly and annual basis. The new public finance accounting system has also been introduced at the regional and federal levels. Although there are some outstanding issues that need to be resolved regarding the use of MOF’s data in RosStat’s compilation of the general government sector into the national accounts 6 , this represents a major accomplishment toward increasing the international comparability of the Russian government finance and national accounts data. Balance of payments. The project was also instrumental in improved the compilation of BoP statistics by supporting (i) the implementation of better data collection on key traded/international services; (ii) introduction of better methods of adjusting foreign trade volume; and (iii) expanding the data collection for computing adjustments to be made for informal trade. Other products. The project also helped to improve the methodologies and techniques for the implementation and analysis of the All Russia Population Census and Agricultural Census in line with international practices and to upgrade the measurement of tourism statistics in line with WTO recommendations and the measurement of unemployment and other labor force statistics in line with ILO recommendations, among others.

4 LFS sample size is not large enough for a reliable breakdown of production by industry and estimates are unstable over time. The results are not consistent with other production methods, but this is not entirely surprising. If results were identical the labor input method would be no more effective than the current methods. 5 The Ministry of Finance (MOF) is responsible for the compilation and analysis of government finance statistics and provides RosStat with GFS data for preparation and presentation national accounts. 6 These include typical coordination issues on disaggregation of current and capital expenditures, reporting on pseudo-government entities, revenues by function and economic activity for market activities of budget-financed entities and state-owned enterprises, disaggregation of transfers and subsidies, all of which affect the compilation of the National Accounts, and valuation and depreciation issues related to fixed assets including government buildings and certain immaterial assets such as mineral exploration.

13

All-Russia Products Classifier by Types of Economic Activities. The development and adoption of a new internationally-comparable product classification (by economic activity) the All-Russia is a major achievement of this project. It is correlated with the earlier All-Russia Classifier of Types of Economic activities, EU statistical classification of production by economic activities and the PRODKOM list. It will allow for the unequivocal identification of goods and services in the market economy, and ensure comparability of production, sales, exports and imports of products internally and internationally, as implementation gradually proceeds. Over time, the impact of this new classifier will spread across a wide range of statistical products. Training. The STASYS project was positively crucial in upgrading the technical capacities of RosStat staff in a broad range of international methodologies and standards. Over 5,500 staff received training, a large number in overseas institutions and from international experts, neither of which was permitted under Russian government rules using RosStat’s own budget. The project has also helped to sustain this effort through the development of an ongoing training plan, as well as the creation of computerized and distance learning training facilities at the central and some regional offices of RosStat. Good quality documentation and training materials have been developed and continue to be in use. On the basis of this example, RosStat’s own funded training has received a significant boost in resources. Although retention of trained staff has been an issue in Moscow and a few other major cities, in regional offices where the labor market is thin this is less of a problem. Meanwhile, the attrition of urban RosStat staff located in big cities to the private sector, research institutions and academia can be viewed as an indirect means of increasing the capacity of users in the RosStat’s products.

(v) Increase the efficiency and productivity of statistical production through improved organization, data collection and processing methods and equipment.

This objective was addressed primarily through two major inter-related efforts. The first was to begin conversion from the Soviet all-census data collection framework to an integrated census-sample survey framework which is more efficient and suitable for a market economy. The second was to complement this effort with the first stage of a modernization program for data collection, organization, processing and dissemination systems in RosStat. The progress on the former was commendable though less than anticipated at project appraisal, whereas on the latter the achievements were substantial and highly satisfactory.

A very thorough report was prepared looking at the potential impact of sampling for all regions and sectors for all surveys and sparked an important debate on the appropriate level of statistical-geographical disaggregation for Russian public policy. This report fed into the development of a systemic plan with timetable for replacing regular economic/business censuses with surveys. The value of using sample surveys to produce reliable national statistics is well-recognized, and their use is gradually being promoted within RosStat. Indeed the agency already uses the results of a number of sample surveys to improve quality of data and better presentation of statistics in publications such as quarterly reports of the labor force statistics, National Accounts, and the Balance of Payments. However, very few additional surveys were actually converted under the project. The new FTP specifies that a percentage of sample surveys will increase 20-30 percent over 2005 levels during the period of the plan, i.e. by 2009 and that surveys will be run to monitor all sectors of the economy.

This lack of conversion is not for lack of methodological skills or resources, but more due to the continuing demand for rayon level data. It should be noted that even in well-developed national statistical offices in Europe and elsewhere which do not collect such detailed data, there is a

14

constant demand for small area data that statisticians are trying to meet (largely unsuccessfully) through modeling. Additional demand in Russia comes from the legacy of Soviet-era economic planning. Nonetheless, the collection and dissemination of such detailed data should be justified from a cost-benefit perspective. There is clearly scope for more conversions in Russia, but this will require careful management of stakeholder expectations.

In preparation, however, key survey infrastructure was established by the project. These include:

• plan and testing of forms for a benchmark Economic Census (which has not yet been completed);

• completion of a Population Census in 2002 and an Agricultural Census in 2006 both with methodological technical assistance (although differing levels) from the project;

• pilot testing including methods for automatic control and editing of data for censuses and sample surveys of individual entrepreneurs and of small enterprises;

• methodological improvements and development of system for automatic editing of raw data for household surveys;

• development of various sampling techniques and frames for various sectors of the economy; and

• substantial training of staff (mentioned above).

The project did make a substantial contribution to modernizing computer facilities of RosStat for survey processing. An Integrated Processing System with 17 subcomponents was developed and successfully tested in 15 pilot regions. Before the project, RosStat was maintaining an ICT infrastructure base that was obsolete. The data processing capability at all levels of RosStat system was poor. The stock of older generation mainframe and mini computers was difficult and expensive to maintain and often unable to run more modern software. Planning of IT investments had not reached the point where the organization was able to execute a system or project requiring special management. Furthermore, the technology architecture did not allow RosStat to take advantage of state-of-the-market, web-enabled technology.

As a result of the project, an information infrastructure foundation (including communications, hardware, application, database, file, mail servers, limited forms management, and software) now exists for implementing a large-scale unified statistical data collection, processing, warehousing and delivery system throughout RosStat. In addition, the project

(i) developed an overall plan and detailed technical design for the development of the entire ICT infrastructure;

(ii) created specialized information software, and subsequently tested and accepted 17 subsystems covering RosStat’s key operations and processes to pilot sites;

(iii) established up-to-date server rooms at the Central Office, the Main Interregional Computer Center, the Research and Design Institute, and in 14 Territorial State Statistics Authorities (TSSAs);

(iv) procured and equipped 6,730 workplaces with modern computers including all workplaces of Central Office employees;

(v) established a high-speed fiber optic communication channel between the Central Office and the Computer Center (Moscow), and Internet and Intranet channels for the pilot facilities;

15

(vi) established training centers with distant learning tools at the Central Office, the Computer Center, and in 6 TSSAs to provide training of RosStat specialists; and

(vii) left in place several hardware and software maintenance contracts essential to keep these products up to date with the latest upgrades, patches, and telephone support to resolve problems.

In addition, technology for efficient warehousing and processing of microdata and aggregated statistical data was developed. Some 300 RosStat specialists were trained in 1-2 week courses on use of new software systems, not including separate training on equipment and e-mail/e-document systems. Additional impact has been gained from (a) an improved staff morale among RosStat because of their being equipped with improved computing tools; (b) a stronger basis for collective work irrespective of geographic location, and (c) a new level of openness engendered by upgrades to RosStat’s website. A key achievement is that subsystems are based on solutions that allow for increased functionality and application at different organizational levels of statistical observation, as well as utilization by mid-level specialists. (vi) Improve the quality of analysis and monitoring of economic development along with the modernization of decision-making methods for economic policy. This project supported the development of a new macroeconomic forecasting model tailored to the Russian economy at the Ministry of Economic Development and Trade (MEDT), which has helped to improve forecasting economic activity and impact of policy reforms. Overseas training was provided to around 40 MEDT staff on forecasting methods and the knowledge gained was used by staff to build the model. After encouraging test results, the model has been used, inter alia, to measure the impact of tax exemptions in government revenue, and the impact of international oil trade on Russian oil prices. Some limitations of the model are that it does not cover production functions and only computes GDP from the demand side because data on production (and investment by enterprises and households) is very weak. Also the model currently uses industrial classifications only, but the MOEDT plans to include functional classifications shortly. However, in building the model, MOEDT has made efforts to harmonize international industrial classifications with EU classifications. The project also supported the identification of a set of indicators to be derived from the SNA93 framework for monitoring of Russian economic activities in the short, medium and long term. Other future activities include the delivery of training to other uses, especially those in the planning units of other line ministries.

3.3 Efficiency The efficiency of the project itself was affected by a number of government wide factors mentioned above in Section 2.2. A number of these resulted in implementation delays, eventually an “over-aged” project and an increase in supervision resources associated with the multiple extensions of the closing date. However, in turn the patience on the part of the Bank to stick with the project as it continued on a slow but steady implementation path eventually paid off. As with other projects in the Russian portfolio that spanned a similar time period, the Bank’s steady partnership has culminated in a renewed interest in its technical support and solid commitment on the part of the authorities to complement this expertise with significant financial resources. The finalization of the Statistical Master Plan, new Federally Targeted Program and STASYS 2 are evidence of this. The new model of lending with smaller financial resources from

16

Bank leveraging and helping to guide a much larger Russian-financed program is likely to be much more efficient in terms of development outcomes for the World Bank. In terms of the efficiency of the statistical system, it is hard to measure the impact of the methodological improvements and increase in staff capacity provided through the project. The efficiencies to be derived from improved information technology in data collection, greater reliability of data, and improved quality of services provided by RosStat are difficult to quantify at this stage, in particular because the modernization effort is not yet complete. However, anecdotal evidence suggests that in the pilot sites more efficient research tools are (a) allowing reductions in the time required to introduce changes into All-Russian Classifiers, (b) significantly reducing the time required to obtain output tables, (c) providing increased information analysis capabilities, and (d) reducing the time and costs of statistical publications. The same is evident in the scope and timeliness of trade statistics produced by the Customs Service.

In the future, RosStat has proposed some of it is own performance measures in the context of Russia’s program budgeting reforms. The new project will encourage the further development, and regular monitoring and analysis of these and other measures of productivity and efficiency at RosStat and in the statistical system as a whole. These should include the usual measures such as data produced per resources invested, but also systematized performance measurement of various information processing systems that are being acquired7. In terms of efficiency gains, there is one area of concern relating to the RosStat’s adjustment of human resources in response to its improved technological capabilities. There will be a need to reconfigure staffing both geographically and functionally in order to fully benefit from productivity gains of the IT modernization. Despite technical assistance provided to strengthen RosStat’s managerial (and technical) capacity in the area of human resource management, its not clear whether active plans are being made to make these adjustments.

3.4 Justification of Overall Outcome Rating Rating: Satisfactory Weighing (a) the continued relevance of the project goals to Russia’s development; (b) the substantial achievements of the project both in planned and unplanned improvements to the state statistical system; (c) the successful transition to what looks to be a more efficient intervention by the Bank in this area in Russia; and (d) against the inefficiency of an over-aged project, the overall outcome is rated as Satisfactory.

7 Some advanced statistical agencies use “Earned Value Management” tools that integrate the investment scope of the IT work with schedule and cost elements for optimum investment planning and control, and risk adjusted life-cycle cost estimations of the overall estimated cost for a particular IT investment, including direct and indirect internal costs plus any period or continuing costs of operation and maintenance. Initial cost estimates for the CIS activities were shown to be correct. Where there were cost savings from certain lots, they were used for additional purchase of IT goods of the same category.

17

3.5 Overarching Themes, Other Outcomes and Impacts (if any, where not previously covered or to amplify discussion above) (a) Poverty Impacts, Gender Aspects, and Social Development The program did not have any direct impact on poverty, gender inequities and imbalances, and social development. However, the project helped strengthen the survey base to monitor poverty and social changes, and the dissemination of survey results to allow independent research on poverty and social issues and policies. (b) Institutional Change/Strengthening In addition to the goals and outcomes discussed in Section 3.1, the project aimed to significantly improve the institutional capacity of the state statistical system, in part, through the passage of the Law on Statistics and strengthening of RosStat as an institution to lead and coordinate the overall system. During preparation and effectiveness, the PHRD grant was instrumental in raising the capacity of the BEA and RosStat in preparing some of the more complex components, including on sampling, computerization and organizational restructuring. Despite a long period of preparation since 2000, the Law on Statistics and subsequent regulations which are expected to, inter alia, reinforce RosStat’s leadership of, and coordination across, the numerous agencies in the statistical system, was only enacted in late 2007 a year after project closure. While this has not had any material impact on the improvement in technical capacity within RosStat, it has continued to limit the agency’s ability to strengthen coordination within the statistical system and to build relationships and improve coordination with other data collecting agencies. The Law will also help to regularize the relationship between RosStats, its respondents and data users, strengthen its access to administrative data from other government agencies, and eliminate duplications and inconsistencies in data products from across the system and raise user confidence. A key delay involved a long period during which the process for preparation, approval and submission of legislation in the Russian Federation was under review. A later delay resulted from the intervening passage of the Law on Information which required changes to the already developed draft Statistics Law. It is clear now that there has been no powerful champion for the law. Meanwhile laws relating to population and agricultural censuses were passed quickly with support from relevant ministries in order to facilitate those exercises. The Law has now been adopted by Parliament in November 2007. With respect to RosStat, itself, the project did support a number of medium and long term strategic plans which fed into the development of a Statistical Master Plan, the Third Federal Target Program Earmarked Plan 2007-09, STASYS 2 and the computerization project. These marked improved capacity by RosStat to undertake its own strategic planning in a number of areas. On the other hand, little organizational restructuring of the state statistical system or RosStat itself has occurred as planned under the project, except as consequence of the state-wide administrative reform. To date, RosStat’s organizational structure appears to have been reorganized only by external fiat of the Government’s public administrative reform effort. No Statistical Commission or other advisory body to provide overall guidance to the system of agencies was established. Several inter-agency working groups were established for coordination of project activities, but have not yet been elevated to a formal coordination body for the statistical system. Continued coordination problems with a number of administrative agencies, including for example, the tax authority are evidence that this area has not been strengthened sufficiently. It has been explicitly stated that internal restructuring is not a goal of the new Third Federal Targeted Program.

18

However, although the project did not succeed in the restructuring of Goskomstat nor go beyond the preparation of the draft Law on Statistics (submitted to Parliament later in October 2007 and adopted in November 2007), it has had a substantial impact on strengthening the technical capacity of the Russian Federation state statistical system through introduction of new methodologies, data collection, processing, analysis and dissemination systems and processes and building capacity and knowledge of the staff in Goskomstat through extensive training on statistical methods, IT and managerial competencies such as HR. The project has also helped to strengthen data collection, etc in the Custom Committee and the Central Bank, Ministry of Finance, and Ministry of Economy, Development and Trade. The introduction of surveys and the related computerization program will have significant impact on the efficiency of data production in the Russian Federation. Trust and confidence in the state statistical information is growing and the demand has been generated for continued improvement. (c) Other Unintended Outcomes and Impacts (Positive or Negative) Given the comprehensive nature of the project, almost all of the outcomes and impacts were intended.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops A stakeholder workshop was conducted in February 2006 along with a mission to PetroStat, RosStat’s regional office in St. Petersburg, to meet with local users (as well as Petrostat). Workshop participants reported that, despite the persistent legacy of distrust in Russian officialdom, confidence in RosStat by users is increasing. However, there remains a need for RosStat to strengthen presentation of sources and publication of metadata, to publicize new analytical capabilities, to focus on public education on statistics in order to reinforce and accelerate this growing confidence, and to routinely investigate major anomalies with other data sources, particularly at the local level. The relationship with users at the regional level in St. Petersburg is quite strong, as evidenced by the regular and deep interaction between PetroStat, local academia and local public administration. A small rudimentary beneficiary survey was conducted among 36 institutional users of Rosstat’s statistical products in late 2006. Since there was no earlier baseline from project preparation, the results (presented in Annex 4) cannot form a basis for evaluating project impact, but only current levels of satisfaction. Overall, the assessment of RosStat as a provider of official statistics is positive with around 85 percent of respondents indicating trust/confidence in RosStat products. Areas noted for improvement include the scope and comprehensiveness of information provided and the ease of data retrieval for analysis.

4. Assessment of Risk to Development Outcome Rating: Low The sustainability of current achievements under STASYS appears highly likely. The outputs are strongly owned by RosStat and other ministries and agencies (GFS, Customs data). The approval and implementation of the Statistical Master Plan, the third Federal Earmarked Program and STASYS 2 are signs of strong government commitment to the continued development of the statistical system. However, long term sustainability of these efforts still depends on (i) strengthening coordination and management of the whole statistical system including the passage of the Law on Statistics that finally was adopted by State Duma in November 2007, (ii) growing public confidence in RosStat and official statistics in general, as well as (iii) the planned,

19

continued modernization of RosStat’s technology and methodologies; and (iv) improvement in the monitoring and evaluation of investments in the system. Upcoming issues such as further development of fiscal federalism, Russia’s need to address growing inequality and ensure development/diversification of the non-oil economy will put additional demands on the statistical system, but also raise awareness among the policy makers of the need for good information. Some risks may come from the outcomes of Russia’s administrative reform efforts.

Sustainability of IT investments depends on adequate operation and maintenance (O&M) of hardware systems and software, and availability of funds to expand beyond the pilot project. It should initially be noted that, at whatever its level of funding in the past decade, RosStat has skillfully and efficiently maintained its technical infrastructure, as evidenced by the fact that equipment has lasted to the end of its depreciation cycle and far beyond. Sustainability of the IT investments has been promoted by keeping down projected operation and maintenance costs through long-term warranty contracts and ensuring funding to roll-out the national system. Detailed design and procurement specifications took into account future O&M costs (5 years for servers and 3 years for workstations, with most equipment still under warranty, but software will require renewal of licenses). As for public funding of the national roll-out, in the follow-on STASYS 2 project, the Ministry of Finance has decided that the Government would finance the equipment related to the roll-out of the CIS and that Bank involvement was not required in this case. The key risk is that RosStat, like other ministries and agencies, is in dire need of personnel, especially highly qualified IT personnel. Staff retention became an issue following training on the use of new information technologies at some pilot sites. This will need to be carefully managed.

5. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues)

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory The Bank performance in preparation is deemed satisfactory. The Bank responded promptly to help the government strengthen its first Federal Earmarked Program by preparing a review paper, holding a seminar with broad stakeholder participation, and formulating a development strategy. The team correctly identified the long term nature of reforming of a statistical system, in particular one of this size and inheritance, and recognized the need for careful prioritization and sequencing. Key adjustments to the authorities’ plans included: (i) broadening the scope to the wider statistical system, i.e. including other government agencies, besides Goskomstat (now RosStat), involved in the production of statistics and emphasizing the need to improve coordination among them; (ii) a focus on building management capacity for the statistical system; (iii) addressing the legal framework; (iv) a more comprehensive reform of data collection and processing methods; and (v) added emphasis on improving dissemination and building relationships with end-users. Moreover, the project was an important innovation in the Bank and provided a launching pad for the development of the Statistical Capacity Building (StatCap) program which has gone on to provide support to a number of countries, worldwide. A few shortcomings of the preparation process are worth noting for the record even though they have not materially affected outcomes. First, the decision to forego a formal Decision Review

20

meeting8 was not in line with standard practice for an initial project of this size. The second shortcoming was the rather late acquisition of a PHRD grant due to a protracted approval process on the GOR side and the subsequent need for an extraordinary extension of the grant beyond the project’s effectiveness date. Third, as noted above, it is not clear whether the full complexity of the modernization program was understood. For this and other reasons the project implementation period should have been longer than the approved four years. Finally, the Bank misinterpreted the commitment of some of the other implementing agencies to various initiatives included on their behalf. (b) Quality of Supervision Rating: Satisfactory The Bank performance in supervision is deemed very, but not highly, satisfactory. Close supervision at the onset of the project helped to resolve some of the early teething pains. A Quality of Supervision Assessment in 2001 under an assessment of Risky Projects rated project supervision as Satisfactory. Regular biannual supervision missions were complemented later by close field-based supervision. The Bank worked closely with the authorities to resolve a number of implementation problems already described in Section 2.2. As with other projects in the Russia program over this same period, the Bank demonstrated appropriate patience in resolving the portfolio-wide issues leading to a renewed interest and new modus operandi for Bank projects in the Russian Federation. STASYS was the first project in the Russia portfolio to propose and implement the new co-financing scheme. The mid-term review was effectively used as an opportunity to formally make adjustments in project structure which have facilitated this completion review. Financial management and procurement reviews have been undertaken with appropriate regularity. Field-based supervision, including of financial management and procurement processes were important factors that helped to ensure the progress toward developmental outcomes. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory

5.2 Borrower Performance (a) Government Performance Rating: Satisfactory The Government’s performance, that is the performance of the Federal Government as a whole, is rated as satisfactory. Inefficiencies in project preparation and implementation and of the delay in achieving the passage of the Law on Statistics must be judged against the very positive achievements in terms of the renewed commitment to, and increased financing for, the development of state statistical system evidenced in recent years. During preparation, there were critical delays in the approval of a PHRD grant by the GOR and in making payments under the Federal Center for Project Financing (FCPF) grant that consumed initial Bank supervision efforts. Early complications at the start-up including legal issues with the implementing agency, delays (now understood to be a sine qua non in Russia) in preparation and effectiveness due to government bureaucratic procedures, shortages of counterpart financing, and finally, the setbacks to project implementation owing to periodic government-wide proceedings all contributed to the eventual result of an over-aged project. Nevertheless, the project achievements remain unique

8 The project was declared fully appraised by the Country Director in an official memo.

21

among the more problematic Russian portfolio of Bank projects, in being almost fully disbursed and followed up by another operation to continue the development of the statistical system. (b) Implementing Agencies Performance Rating: Satisfactory On the other hand, the performance of the implementing agencies is rated fully satisfactory in light of the project achievements, sustained commitment and effective use of project resources. Implementing Agency Performance Federal State Statistical Service (RosStat) RosStat’s performance is also deemed satisfactory on account of its effective use of project resources to strengthen staff capacity, statistical methodologies and information technology. However, its own inability to undertake its own organizational restructuring, to push for greater leadership and coordination within the statistical system are a notable, but not excessively so, shortcoming of this performance.9

Satisfactory

Bureau of Economic Analysis The BEA provided project implementation management services for STASYS, mainly essential administrative support – financial management, procurement coordination, reporting processes, secretariat for the Inter-Agency Coordinating Committee and Component working groups. The Bureau did not provide substantive oversight and planning. Instead, key substantive choices regarding the project activities were adequately guided by the technical working groups, staff in RosStat and ICC. Notwithstanding the critical start-up delays in particular with adequate and appropriately skilled staffing even though the Bureau had prior experience with Bank projects, the overall performance of the Bureau was very satisfactory, and its performance in the area of IT procurement was highly satisfactory. In the latter regard, BEA provided critical and substantial value to the government during the procurement of both equipment and software. BEA contributed in particular to development of detailed design and bidding documents through qualified IT specialists. It ensured bidding documents were of high quality and understandable to third parties. This in turn led to increased predictability of RFPs and bid proposals.

Satisfactory

(c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory

9 For example, RosStat has not taken any lead in the resolution of the Central Bank’s limited access to the new functionality available in the electronic interface with the Customs data, and at the time of the ICR mission, there was an implicit impasse between MOF and RosStat on which organization was to initiate preparation of a memorandum of understanding on electronic access to the GFS.

22

6. Lessons Learned Statistical systems, like information systems, are complex entities for which development requires strong coordination of progress on many inter-related constituent parts. STASYS was essentially a greenfield effort in which there were a myriad of areas to improve statistical methodologies. As such, the sometimes opportunistic approach to using project resources for upgrading statistical methods remained effective. However, as we move into STASYS 2, the coordination of various efforts to improve different parts of the system of national accounts will become increasingly important. For example, delays in implementing the new Classification of Products will affect the pace of strengthening the compilation of National Accounts. The conversion from census-only to sample surveys in transition country statistical systems is not just technical issue – there are quasi-political considerations. The move from census to sample data will essentially affect statistics users at a fine level of disaggregation. While it is inherently more efficient for a market economy, even developed country statistical systems face continued demand for this type of data, which they try to satisfy by modeling. As such the conversion from an all-census to integrated census-sampling framework in transition countries requires close attention to dealing with concerns of local users or federal users interested in local area disaggregation to make the case of system-wide efficiency as an equally relevant goal. Attention to respondent burden is also critical in the transition from census-only to sample surveys. There is the general impression among RosStat staff that enterprises in the RF are becoming more resistant to providing data. Large and medium enterprises have good response rates, but there are continuing concerns about the quality of data provided. By law, small enterprises receive shortened questionnaires. Their response rates are reportedly on the order of 75 percent. In order to raise response rates and quality of data, statistical systems in transition must develop a systematic way of measuring and containing respondent burden if they are to assure the quality of the sample surveys. Pilot survey of individual entrepreneurs have indicated that face-to-face interviews are a much more effective way of collecting data than self completed mail-out mail-back interviews or telephone interviews. The passage of a framework law governing official statistics is not only crucial for legal reasons but to empower leadership and coordination in the statistical system. The statistical system is as dependent on administrative data as it is on its own primary data collection methods. The absence of an identified body to lead/coordinate the system and certify the quality of a variety of official statistical data creates a vacuum in which distrust and lack of confidence in official statistics could grow. One lesson from this project is that in transition countries, legal basis is not only a critical form of empowerment for the national statistical agency, but its absence can also delay implementation of reforms, including those related to user and respondent relations. Time allowances should be provided for unforeseeable delays and project risks related to the untimely fulfillment of interrelated operations should be considered during project preparation. An important lesson learned in the project pertains to the complex phasing and timing of IT modernization processes. First, the coordinating of the multiple facets of IT modernization, e.g. equipment delivery, software testing, network installation, and civil works for computing centers, especially in the diverse all Russia regional frameworks (as RosStat has) is complex, requires closer monitoring and allowances for unforeseen delays. In one case, various delays in the delivery of equipment, installation of networks, approvals for the remodeling of the server room resulted in an 18-month lag between when the statistical applications were developed and when they could be piloted. Relatedly, software contracts require more realistic contractual periods. In

23

general, time allowances should be provided for unforeseeable delays and project risks related to the untimely fulfillment of interrelated operations should be considered during project preparation. A second and positive lesson from the IT experience under STASYS was the prudence of both a systems-design, and a participatory and step-by-step approach taken by RosStat. At the outset, RosStat approved a comprehensive framework of the “Technical Project for Computerized Network Infrastructure,” approved by RosStat through a consensus. The scope of this framework included computer equipment, data storage, printers and office equipment, operational systems, software and licenses, telecommunications equipment and lease of communication channels, installation work, testing, and training. All investments in IT are considered within this framework and RosStat seeks to reach a balance between increasing computing resources and improving data availability. The participatory approach helped to ensuring the choices made matched the real requirements of the end user and co-opted them into the implementation early on. Finally, in going step-by-step both conformed with international standards and technical solutions and allowed for learning along the way. Finally, the use of an external agency as the PIU can still be an appropriate model provided the main beneficiary is fully committed and engaged in project activities. The standard caution against using PIU remains the foregone opportunity to build internal project management capacity. However in projects like STASYS and STASYS 2, in which several of the components or subcomponents are mini-projects (computer modernization, transition to surveys), the management of these inside the beneficiary institution can substitute more effectively as a learning experience, while leaving the host of administrative details to an external agency. This is clearly not a model for all, but could still remain an option for some project.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies Ministry of Finance of the Russian Federation in its letter as of December 19, 2007 #04-04-09/9090 to the Country Director for Russia advised that it had no comments or suggestions regarding the ICR and performance of the State Statistical System Project. The Rosstat in its letter as of November 12, 2007 #ВС-05-38/3169 informed the World Bank Moscow office it had no comments on ICR. (b) Co-financiers In consultations with the authorities from Japan Embassy in Russia, it was indicated that they found the report in general acceptable. All comments on the PHRD grant contributions were adequately incorporated into the report. Subsequent communications in writing are filed in the project folder in IRIS. (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society)

24

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in US$ million equivalent) Project costs Loan financing

Components

Appraisal Estimate (US$m)

Actual/ Latest

Estimate (US$m)

Percent of

Appraisal

Appraisal Estimate (US$m)

Actual/ Latest

Estimate (US$m)

Percent of

Appraisal1. Improvement of Organization, Management, Strategy, and Perspective Planning 2.24 4.21 188% 2.00 1.98 99% 2. Fostering Knowledge of Standards and Methodology 2.43 4.20 173% 2.17 2.45 113% 3. Strengthening of Statistical Infrastructure 3.86 1.80 47% 3.00 0.66 22% 4. Reorganization of Data Collection, Processing, and Dissemination 20.78 37.78 182% 15.25 19.77 130% 5. Improvement of Government Finance Statistics and the Compilation of the General Government Sector Accounts in the SNA 0.73 0.46 63% 0.60 0.22 37% 6. Introduction and Compilation of Financial Statistics 0.38 0.00 0% 0.35 0.00 0% 7. Improvement of Coverage and Quality of BoP Statistics 2.10 0.34 16% 1.60 0.21 13% 8. Improvement of the Capabilities of the Government in Utilization of Statistical Information for Economic Monitoring, Forecasting 1.28 1.42 111% 1.00 1.12 112% 9. Improvement of Vital Statistics Processing 1.68 0.13 8% 1.50 0.06 4% 10. Establishment of Directorate to Support Project Management 2.70 2.43 90% 2.15 1.38 64% 11.Improvement of Collection and Dissemination of Custom Statistics 0.00 1.72 0.00 1.85

Total Baseline Cost 38.18 54.49 143% 29.62 29.70 100% Unallocated 0.08 0.00 0.08 0.00 Front End Fee 0.30 0.30 0.30 0.30

Total Financing Required 38.56 54.79 30.00 30.00 100%

(b) Financing Source of

Funds Type of

Cofinancing Appraisal Estimate

(USD M) Actual/Latest Estimate

(USD M) Percentage of

Appraisal Borrower 8.50 24.80 292% IBRD 30.00 30.00 100%

25

Annex 2. Outputs by Component Performance Indicators Outputs Impact

1. Improvement of Organization, Management, Strategy, and Perspective Planning KPIs: State statistical institutions have

initiated operations in policy regulations, planning, programming and HR development.

Organizational structure of Goskomstat

is modified and management capabilities are improved

SSS is restructured and strengthened Management skills are enhanced Human resource capacity is built Component outputs: Statistics regulations are issued. Inter-Agency Council and National

Statistical Advisory Board are established.

Federal Statistical Office is restructured. Senior and middle level officials are

trained. Goskomstat personnel database is

established. Human Resource staff are trained. Loan agreement: Law on statistics is adopted. Coordinating Committee and National

Statistical Advisory Board are established.

Federal Statistical Office is restructured. Goskomstat personnel database is

Outputs: • Development of medium and long term strategic plans for the

statistical system, including Statistical Master Plan and Third Federal Earmarked Plan, which are guiding future activities of the Government and support from the Bank

• Development of technical development strategy, plans and simulations for computerized system for data collection, processing, storage, analysis and dissemination

• Development and introduction of new central computer system for RosStat, including email and e-document filing (for entire federal system), and fiber-optic communication between central office and off-site computer center.

• Improved capacity for HR management at RosStat, including updated procedures and processes, and development and introduction of operational HRMIS.

Inputs: • Restructuring the State Statistical System and Improving its

functioning. • Elaboration of a Medium-Term Strategic Plan for Statistical

System Development. • Preparation of a Long-Term Development Program for the

Federal Statistical System. • Development of Human Resources and Personnel-related

Processes. • Introduction of automated HR management system, computers

and other equipment. • Finalization of the Technical Development Strategy for

Goskomstat. • Technical project for development of the computer information

system.

Note: Enactment of a new law and regulations were moved to Component 3 and enhancement of management skills were covered under Component 2.

A number of medium and long term strategic plans were produced feeding into the development of a Statistical Master Plan, Third Federal Target Program Earmarked Plan 2007-09 and STASYS 2. These marked improved capacity by RosStat to undertake strategic and perspective planning.

On the other hand, little organizational restructuring of the State Statistical System or RosStat itself has occurred, except as a consequence of the state-wide administrative reform. No Statistical Commission or other advisory body to provider overall guidance to the system of agencies was ever established despite various plans. Inter-agency working groups were established for coordination of project activities, but none of them has yet been elevated to a formal coordination body for the statistical system.

Capacity of human resource management was improved, both through revised procedures, new IT system and training

26

Performance Indicators Outputs Impact established.

Triggers for follow-on project: State

statistical institution haves initiated modification in policy regulations, planning, programming and human resource development.

• Dynamic simulation of computerized information system. • Development and implementation of the integrated statistical

data collection, processing, storage and presentation system based on the unified forms.

• Computer equipment and software for implementation of electronic mail system and documents workflow system RosStat.

(provided under Component 3).

The project did also provide support to strategic planning for technological development of RosStat which has fed into the implementation of pilots and now rollout of a massive modernization program.

2. Fostering Knowledge of Standards and Methodology KPIs: Staff from data producing

agencies along with users and suppliers have knowledge of standards and methodology.

Capacity to analyze statistics in the

macroeconomic framework are increased.

Staff are trained and retrained in

international methodology for official surveys.

Staff are trained in statistical software. Computerized training facilities are

provided. Component outputs: Goskomstat training needs are assessed

and appropriate plan is completed. Goskomstat headquarters and regional

staff training proceeds according to the plan.

Computerized training facilities are

Outputs: • Computerized training facilities at central and regional offices

of RosStat. • Improved technical capacity over 5,500 RosStat staff on a

variety of statistical and IT competencies. • Preparation and ongoing implementation of a training plan for

RosStat. • Improved capacity in HR management for RosStat managers.

Inputs: • Training assessment and development of a training plan for

RosStat. • Preparation and piloting of a distance learning strategy. • Development of technical documentation for construction,

installation and repair works for computer center, library and training center.

• Assessment of equipment needs, and provision of computers and other equipment to implement training facilities.

• Training of a wide variety of topics, a sample of which are provided below.

Training on international statistical methodologies • Training in the U.S. on Statistical Issues of Food-Intake

Estimation

The training program financed by the project has been well executed with over 5,500 staff being trained in new methodologies including for sample surveys, and in new information technology. Good quality documentation and training materials have been prepared and continue to be used. In addition, it is important to note that RosStat’s own funded training has continued in parallel to the project and has received a significant boost in resources over the past few years, e.g. from R4 million in 2005 to R27 million in 2006.

A number of training courses have also been held for senior officials to build management capacity.

One key benefit for the project and World Bank financing was training abroad and by international experts in Russia (neither of which are permitted using RosStat’s own budget).

27

Performance Indicators Outputs Impact being established in Goskomstat.

Triggers for follow-on project: The

staff of the state statistical institution along with users and suppliers have knowledge of standards and methodology.

• Training in the U.S. on Methodological and Organization Principles the "Input-Output" Tables Development.

• Methodology of Forming Certain Price and Financial Statistical Values (distance training).

• Training "Development of national accounts system". • Organization of statistical observation in construction sector. • Participation in 55th session of International Statistical Institute

Training in statistics, software and other subjects required to support new survey work • Training "Methods of Statistical Surveys Conducting". • Training in the U.S. on Remote Collection of Data as Part of

the Employment Survey. Training in information technologies including in pilot regions. • Training on the Organization of Preparation and

Implementation of Computerized Processing of Agricultural Census Data.

• Use of the SPSS for Windows and SPSS Samples software • Working with subsystem of statistical data collection and

processing. • Organization of Maintaining a Statistical Register on the Basis

of Interdepartmental Information Interaction Using the “One Window” Technology (distance training).

• Training in Human Resources Automated Management Systems.

Management training (from Component 1) • Organization of statistical activity and management in national

statistical offices in EU countries. • Training "Organization of statistical activity and management

in Russian Federation" (MESI).

The project did support the establishment of computerized training facilities and initiated the development and use of distance training facilities in certain regions. These to be rolled out into regional training centers as fast as the network development will allow.

Retention of trained staff at headquarters and in large cities continues to be an issue, mainly among younger employees, while in other regions and local offices this is much less of a problem. One unintended impact of the departure of trained staff is that for the most part they remain in Russia and have formed a cadre of statistical analysts working in other parts of the government, in academia or research organizations or in the private sector as informed users of RosStat outputs.

28

Performance Indicators Outputs Impact

3. Strengthening of Statistical Infrastructure KPIs: Basic infrastructure for business

registers, standard classification and surveys are supported by newly acquired computer systems.

Infrastructure for survey

implementation is created. Sound relations between statistical

organizations and stakeholders are established.

Managerial and planning facilities are

provided. Survey design and operational facilities

are established. General household social survey facility

is established. Standard classification unit is

established. Component outputs: New and strengthened linkages are

established. Strategic planning process is established. Survey design process is developed. Master sampling scheme is developed Classifications unit is operational. Loan agreement: Business register is updated in a timely

Outputs: • Draft law on statistics. • Development and adoption of internationally compatible All-

Russian classification of products by economic activity. • A wide range of improved statistical methodologies in the

compilation of national accounts. • Development of several new statistical products related, inter

alia, to tourism, innovation and competitiveness, structure of labor force and regional development.

Inputs: Legislation • Development of regulatory and legal documents supporting

official statistical activities in accordance with the existing legislation of the Russian Federation and recommendations of international organizations.

• Development of the draft of the Federal law "The official statistical account and the state system of statistics in the Russian Federation".

Classification systems • Development and implementation of All-Russian Products

Classifier by Types of Economic Activities correlated with All-Russian Classifier of Types of Economic Activities, EU statistical classification of production by economic activities and the PRODKOM List.

New methodological approaches • Introduction of commodity flow method. • Improvement of calculation techniques for obtaining dynamic

GDP estimates on the basis of supply and use tables for goods and services.

• Development of methodology for statistical observation over the concentration of production of main groups of industrial

Note: all activities pertaining to introduction of survey methods were consolidated under Component 4 (reviewed below).

Legislation & stakeholder relationships Despite a long process of preparation and revision since 2000, the Law on Statistics and subsequent regulations were not enacted during the project implementation period. While this has not had any material impact on the improvement in capacity within RosStat, it has continued to limit the agency’s ability to strengthen coordination within the statistical system and to build relationships and improve coordination with other data collecting agencies. The Law has been adopted by the State Duma in November 2007.

Other than (i) the establishment of several technical advisory committees including non-governmental expert users (academics, researchers) for select statistical products; (ii) a workshop; and (iii) a small ad hoc and rudimentary survey for this report, there were no overarching changes to the relationship between RosStat, the broader statistical system and non-government users.

In addition, no activities were financed geared at improving feedback from or measuring burdens on survey

29

Performance Indicators Outputs Impact manner and is fully operational in a LAN environment.

Sample survey design process is developed and adopted.

Master sampling scheme is developed.

substitute goods. • Development of the methodology for recording investments in

non-financial assets, compilation of the balance sheet for non-financial assets.

• Methodology and information support for compilation of balances of food resources in compliance with international standards.

• Improving the methodology for national accounts and input/output tables compilation.

• Improving compilation of national accounts on the basis of classification standards development.

• Improving the input/output tables compilation based on classifications standards development.

• Development of the methodology for assessing current market value of fixed capital using the perpetual inventory method.

• Development of methodological recommendations for "cost-of-living index" calculation.

• Translation of the OECD Manual “Measuring Capital”. New statistical products or other improved methods • Aggregate index of economic development at sub-regional

level. • New methodologies for implementing recommendations of the

World Tourism Organization on tourism statistics in Russia. • Development of information and statistical database for the

assessment of national economic potential for innovation. • Development of the methods of analysis of the professional

and qualification structure of the labor force based on recommendation of international organizations.

• Development of system of parameters of competitiveness of the Russian economy.

• International expert evaluation of methodology for statistical observation and calculation of indicators used to estimate the unemployment rate in line with the ILO definitions.

• Concept for the organization of performance monitoring

respondents.

Introduction of new classification The development and adoption of a new classification system for products by economy activity is a major statistical development in Russia that will allow internationally-comparable tabulation of the SNA for the first time. Over time the impact of this new classifier will spread across many statistical products. Methodological improvements to SNA The project supported significant improvements to the compilation of national accounts toward compliance with SNA 93 international standards. These included: • Introducing the commodity-flow

method; • Improvements in using supply and use

tables based on this method for GDP estimates;

• A method of recording investment in non-financial assets;

• A method of compiling tourism statistics; and

• Classification of flows and stocks.

Future plans include the implementation of the full sequence of accounts and balance sheets articulated in SNA 93. New statistical products. The project also supported the development of new statistical products to

30

Performance Indicators Outputs Impact indicators of government agencies and monitoring of performance indicators for budgetary planning units.

support analysis in key areas of policy making, including performance indicators for government budgeting, regional development, competitiveness and innovation.

4. Reorganization of Data Collection, Processing, and Dissemination KPIs: State statistical bodies understand

the concepts and standards of market-oriented data collection, processing and dissemination, and have begun modernizing their publishing and printing infrastructure.

PAD: Data collection system is

reorganized. Process of replacing regular economic

business censuses with sample surveys for monthly and quarterly statistics are developed.

Specialist sample survey design work is expanded and reorganized.

Sample survey statistics with specification of sample error possibilities are presented.

Periodic benchmark censuses of economic activity are conducted.

Techniques of updating last-stage serial sampling units are reviewed and improved.

Data collection methods are improved. Questionnaire design is reviewed. Computer facilities for survey processing

are acquired. Statistical applications software are

acquired.

Outputs: • Development of integrated census and sample survey

framework. • Partial implementation of plan to replace economic/business

censuses with surveys, including development of sampling frames, re-organization of data collection and processing, pilot testing of new methods, and techniques, software and hardware for automatic control and editing of census and survey data – key areas include individual entrepreneurs.

• Development and pilot of modernized data collection and processing system.

• Methodological improvements, modernization of data processing and new public on-line access to results from Household Budget Survey.

• Technical and methodological Support to the implementation of the All Russia Agricultural Census.

• Development of RosStat website for data dissemination (Component 1).

• Implementation of system for GIS analysis and dissemination of data.

Inputs: • Replacement of economic business censuses with surveys. • Development of a systematic plan, with timetable, to complete

the process of replacing regular economic/business censuses. • Development of a schedule for the transfer of statistical

observation of economic units to a sampling method. • Draft project of Economic Censuses in the Russian Federation • Testing of forms for the Economic Census.

Replacement of economic business censuses with surveys Excellent preparation has been made to transit from censuses to surveys in a number of areas, including development of sampling frames, re-organization of data collection and processing methods, pilot testing of new methods, and techniques, software and hardware for automatic control and editing of census and survey data. However, while the Population and Agricultural Census have been undertaken, the Economic Census has experienced critical delays. Meanwhile only 5 surveys have been converted from full coverage to sample surveys as direct consequences of this project. Some other surveys were converted by RosStat to sample surveys before the Project This lack of conversion mainly due to the continuing demand for rayon-level data, which is not entirely unjustified at a time an administrative reform is giving more responsibilities to local government.

Improvement in household surveys. The project, along with RosStat’s

31

Performance Indicators Outputs Impact Data dissemination process is

reorganized and modernized. Component outputs:: Plan with timetable is completed. Revised scheme is developed and

implemented. New census standard is developed. Survey on sample updating is completed

and recommendations implemented. Data-collection review is finished. Questionnaire review is completed. Equipment is acquired. Training in equipment is provided. Statistical software is applied Goskomstat publishing office is

modernized. Training in dissemination is completed. Loan agreement: Master program for

data collection is developed and approved.

Survey on sample updating is completed and recommendations are implemented.

Data collection review is finished. Questionnaire review is completed. Computer equipment for regional

processing centers and headquarters is procured.

IT and network infrastructure are upgraded based on replication of pilot projects.

Training in utilization of computer equipment is provided.

• Organization of All-Russia Agricultural Census based on FAO UN program.

• Methodology, organization, pilot testing, and development of methods for automatic control and editing of data for censuses and sample surveys for individual entrepreneurs.

• Finalizing public relations strategy and pilot testing survey of small enterprises.

• Improvement of statistical observations in construction. • Transition to new methods of designing samples frames for

observing price behavior in the industrial sector. • Improvement of sample frames and programs for analytical

surveys of enterprises and organizations from various sectors of economy in line with EC recommendations.

Improvement in general household surveys • Development of a system for automatic editing of the

Household Budget Survey (HBS) raw data. • Methodological recommendations for the program of

household income surveys. • Creation of open access system to HBS information.

Reorganization of data processing • Specialized statistical software, including SPSS. • Software development of information-analytical system based

on integrated database with 17 subcomponents pilot tested in XX regions that processes a “P suite” of forms, including a two-level integrated system of data processing to obtain national accounts estimates.

• Computer equipment and software to allow open access to micro databases, including HBS (see above).

• Mini-computers for transition to electronic data collection on price levels in regional and territorial units.

• Acquisition of automated edit and imputation software system (CANCEIS).

• Development of TOR for partial technical survey of RosStat

collaboration with the World Bank’s poverty assessment work, have improved the quality of the household surveys. Both were also instrumental in creating open access (through CDs) for researchers to the observation level data.

Modernization of data dissemination An information infrastructure foundation (including communications, hardware. application, database, file, mail servers, limited forms management, and software) now exists for implementing a large-scale unified statistical data collection, processing, warehousing and delivery system throughout RosStat.

32

Performance Indicators Outputs Impact Triggers for follow-on project: The

state statistical bodies understand the concepts and standards of market-oriented data collection, processing and dissemination, and have begun modernizing their publishing and printing infrastructure.

headquarters and expert assessment of survey results. • Delivery and installation of computers, other equipment and

software to implement, pilot and replicate technical solutions for data collection, processing and dissemination including:

o Modern computing equipment for over 11,000 (or 45 percent of RosStat) workstations.

• Development of network infrastructure and server units for RosStat headquarters, Computer Center, training institute and 14 regional offices.

Modernization of data dissemination • Development of RosStat website (Component 1). • Development and implementation of a program for GIS-based

dissemination of statistical data, including computer, other equipment and software.

5. Improvement of Government Finance Statistics and the Compilation of the General Government Sector Accounts in the SNA KPI: General government accounts are compiled. Component outputs: Economic and financial classifications of government accounts are consolidated and enhanced. Sequence of accounts for General Government scheme is developed and staff trained. Triggers for follow-on project: Government finance processes for local government accounts, sequence of accounts and debt statistics initiated.

Outputs: • Support to the development and production of GFS2001 and

COFOG compliant government financial statistics, including at regional and federal levels.

• Improvement in the complication of general government sector accounts in the system of national accounts (SNA).

Inputs: • Development of mechanism of implantation of integrated

accounting program of the sector of state management and instructions for its application.

• Training for staff at Statistics Norway, at Statistics Australia and in the Australian Treasury.

The project was instrumental in supporting the conversion of budgetary classification and accounting methods to allow compilation of GFS2001 and COFOG compliant government financial statistics, a major achievement of the Ministry of Finance in 2005. Beginning January 2006, MOF has been able to compile internationally comparable government finance statistics at federal and regional levels. A few methodological issues remain to be resolved for accurate compilation of national accounts, including: (i) coverage of all sub-sectors of general government including extra-budgetary agencies, e.g.

33

Performance Indicators Outputs Impact Pension fund, and profit-making state-owned enterprises; and (ii) treatment of subsidies and current and capital transfers.

6. Introduction and Compilation of Financial Statistics KPI: Collection of basic financial

statistics (beyond money and banking) is operational.

Financial statistics are compiled Component outputs: Coordination mechanism is established. Study on financial register is completed. Data collection manual and training are

completed. Legal Agreement: Mechanisms to

collect statistics on non-banking financial enterprises is established.

Triggers for follow-on project: Basic financial statistics (beyond money and banking) system is initiated with mechanisms for a financial register, coordination and data collection.

No inputs financed or outputs produced by the project. After some time and effort to engage the Central Bank on this area, this Component was dropped.

7. Improvement in the Coverage and Quality of Balance of Payments Statistics KPI: BoP statistics system

(responsibilities, data collection methods, customs statistics and data transfer processes) are restructured.

BoP statistics are improved. Component outputs: Responsibilities are restructured.

Outputs: • Improved compilation of BoP statistics through (i) stronger

data collection on key traded/international services; (ii) better methods of adjusting foreign trade volume; and (iii) expanded data collection for computing adjustments to be made for informal trade.

Inputs: • Consulting service to develop organizational and

This Component was largely achieved. The BoP work was implemented by the Central Bank whose responsibilities for compilation of BoP statistics has been clearly outlined for some time A number of methodological improvements were made and key adjusters (for informal trade – a key issue in Russia) were

34

Performance Indicators Outputs Impact BoP transfer data is improved. System to collect and process data on

foreign trade in services is established and operational.

Triggers for follow-on project: BoP

statistics data collection is strengthened at all levels, methodologies for cross checks and adjustments are developed and implemented, and data sharing processes are restructured.

methodological approaches to improve techniques for adjusting foreign trade volume.

• Consulting services to develop institutional and methodological approaches for improving collection of data on imports of services by defining implementation arrangements for benchmark and on-going sample surveys.

• Consulting services to develop methodology and conduct calculations to define the difference between CIF and FOB prices for imported goods.

• Pilot survey of sales volume for consumer goods at manufactured goods city markets.

• Surveys of foreign transportation companies’ representatives and Russian importers of construction services.

estimated based on sample surveys. However, at the time of this report it is not clear whether these surveys are to be repeated and the periodicity of them. This may reflect the continued lack of convening power of the RosStat over the larger statistical system and ability to ensure quality of constituent statistical products of other agencies. Work on customs statistics, including the provision of computer equipment and training, was separated into the new Component 11, reported on below.

8. Capacity Building in Use of Statistical Information for Economic Monitoring, Forecasting KPI: MOE is expanding the list of

indicators used for economic monitoring and applying in practice models of economic forecasting.

Economic monitoring, forecasting and

policy making are improved. Legal Agreement: Forecasting models

appropriate to Russian economy are developed.

Computer equipment and software for the economic forecasting is procured and staff trained.

Triggers for follow-on project: MoEDT is ensuring the expansion of the

indicators used for economic monitoring and is pilot testing new

Outputs: • Completion of a new suite of macroeconomic forecasting

models, including testing, manuals and improved staff capacity to design, adapt and use these models.

• Central statistical database established at MoEDT for use in these models.

• Development and adoption of a internationally-comparable All Russia Classification for Products by Economic Activity.

Inputs: • Creation of a Central Statistical Data Bank (CSDB) for

MoEDT and training on its use. • Consulting services to improve Economic Modeling of the

Russian Economy • Overseas training for around 40 MoEDT staff and selected staff

of private forecasting companies on the development and use of the model.

• Software, computers, and other equipment for the MoEDT

The development and adoption of a new internationally-comparable product classification (by economic activity) for the Russian Federation is a major achievement of this project. It will allow unequivocal identification of goods and services in the market economy, and ensure comparability of production, sales, exports and imports of products internally and internationally, as implementation gradually proceeds. This Component has helped to improve forecasting economic activity and impact of policy reforms for the Russian Economy. After encouraging test results, the model has been used, inter alia, to measure the impact of tax exemptions in

35

Performance Indicators Outputs Impact models of economic forecasting. government revenue, and the impact of

international oil trade on Russian oil prices. The project also supported the identification of a set of indicators to be derived from the 93 SNA framework for monitoring of Russian economic activities in the short, medium and long term. The model does have some limitations, namely, it does not cover production functions, and therefore only computes GDP from the demand side. This is because data on production is very weak, especially data on capital investment by household and the enterprise sectors. The model uses industrial classifications only, but the MOEDT plans to include functional classifications in the model.

9. Improvement of Vital Statistics Processing KPI: Vital statistics are processed with

new equipment and software and processing system is strengthened.

Triggers for follow on project: Local

register offices (ZAGs are inputting vital statistics electronically with new equipment and software in pilot regions.

Outputs: • Technical assistance to improve the methodologies for vital

statistics. • Technical assistance for the implementation and analysis of the

2002 All Russia Population Censuses. • Dissemination of results and experiences from the 2002

Population Census. Inputs: • Development of methodological guidelines on Vital Statistics. • Training provided to staff on "Improvement of Vital Statistics"

at the Office for National Statistics of Great Britain.

Original outputs were curtailed and as a result, neither the original KPI nor Triggers for follow-on project were achieved. However, the project did provide key technical assistance to the 2002 All Russia Population Census which was instrumental in improving the implementation, analysis and dissemination of the results as well as the

36

Performance Indicators Outputs Impact • Development of classifiers for creating and recording of

primary non-personal vital statistics data. • Study of the French experience in conducting the All-France

Population Census in 1999. • Study of international experience in organizing and conducting

of other population censuses. • Methodological recommendations for analyzing the 2002

Census Results and the use of census results to improve Russian population statistics.

• System of probabilistic population forecasting development • Methodological recommendations for appraisal of age and sex

population structure of Russian Federation territories. • Adaptation of software for use in analyzing population census

results. • Development of a monitoring system of population health and

medical service delivery in Russia. • Methodological recommendations to improve production of

statistical data on the natural population movements. • Translation, editing and publication of materials of the

international symposium "Russians in a mirror of statistics: the All-Russia Population Census of 2002".

creation of new statistical products, e.g. natural population movements and causes of death (a key concern in Russia today).

10. Establishment of Directorate to Support Project Management (reported on above)

11.Improvement of Collection and Dissemination of Custom Statistics Customs-based foreign trade statistics is

improved (adjusted) and accessible on-line [Loan agreement]

Outputs: • Computerization and expansion of data analysis and

dissemination, with on-line access to off-site state users. • Development and production of new statistical series.

Inputs: • Development of technical requirements for information system

providing access to customs statistics data. • Training, computers and other equipment to implement the new

system.

The new system, which received a technical award from the Russian IT community, has greatly improved the statistical system’s ability to compile statistics on imports and exports of goods, the coverage and timeliness of, and access to, statistics related to balance of trade. However, the CBR which is officially responsible for compiling BoP statistics, is unable to download and print information from the system -- the

37

Performance Indicators Outputs Impact Ministry of Finance and RosStat are the only agencies (outside the Customs Service) with this level of access. This ultimately limits the timely provision of BoP data to RosStat for compilation of the national accounts. Further improvements to address this are planned for the new project.

38

Annex 3. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members Names Title Unit Responsibility/Specialty

Lending Paul S. Armington Principal Economist AFTPR Peer Reviewer Misha V. Belkindas Manager DECDG Team Leader

Jeanine Braithwaite Senior Social Protection Economist HDNSP Peer Reviewer

Rama C. Chandran Consultant ECSPS Procurement

Steve Kuzmicich Consultant ECSPS Statistical System Development

Chandrakant Patel Consultant ECSPS Statistical System Development

Mustafa Dinc Economist/Statistician DECDG Statistical System Devt Olga V. Ivanova Consultant DECDG Statistical System Devt

Andrew James Mackie Consultant ECSPS Fin Mgmt & Disbursement

Tserendagva Odongua Consultant DECDG Consultant Gennady Pilch Sr Counsel LEGEC Legal Olga Vadimovna Shabalina Economist DECDG Project Coordinator

Gurcharan Singh Sr Procurement Spec. ECSPS Procurement Supervision/ICR Misha V. Belkindas Manager DECDG Team Leader Michael Colledge Consultant DECDG Statistical System Devt Mohammed Omar Hadi

Senior Program Assistant DECDG Team Assistant

Camille Anne Nuamah Senior Economist ECSPE ICR Team Leader Lucas A. Ojiambo Senior Economist AFTP2 Statistical System Devt Olga Vadimovna Shabalina Economist DECDG Project Coordinator

Alexander Balakov Procurement Specialist ECSPS Procurement

Galina Kuznetsova Sr Financial Management Specialist ECSPS Financial Management

Aziz Mamatov ET Consultant ECSPS Financial Management

39

b) Staff Time and Cost Staff Time and Cost (Bank Budget Only)

Stage of Project Cycle No. of staff weeks

USD Thousands (including travel and

consultant costs) Lending

FY98 149.37 FY99 118.50

Total: n.a. 267.87 Supervision/ICR

FY00 40 126.03 FY01 32 119.18 FY02 20 108.94 FY03 30 105.71 FY04 35 108.18 FY05 35 114.38 FY06 33 150.80 FY07 13 56.30

Total: 228 889.51

40

Annex 4. Beneficiary Survey Results In late 2006, RosStat conducted a small beneficiary study (for this report) to gauge the current levels of satisfaction with its published statistical outputs. The questionnaire below was sent to a broad range of institutional users of statistics 10 including government bodies and research institutes. Table 1 (below) present the results. Since there was no baseline against which to compare these results, they cannot form a basis for evaluating project impact, but only current levels of satisfaction. The rudimentary nature of this survey is an indication that RosStat still has a far way to go in terms of internalizing the need for better relations with the whole spectrum of users as a means of improving the use of its products. The overall assessment of RosStat as a data provider, among survey respondents, is positive, with around 85 percent of respondents indicating trust/confidence in the official statistical information. However, it terms of dissemination, very few respondents were satisfied with the scope (15 percent only) and comprehensiveness (5 percent) of the statistical information provided on the website. A majority were also dissatisfied with the ease of information retrieval for analysis.

Table 1. User Survey Results # Question Response No. of responses %

Hard copy 14 Website 20 1

How do you prefer to receive the official statistical information from

RosStat? CD rom 21 Are you satisfied with:

yes 5 15.2 no 26 78.8 2

The scope of information being published in RosStat official

statistical sources? hard to say 2 6.1 yes 14 42.4 no 16 48.5 3 The timeliness of RosStat’s provision

of official statistical information? hard to say 3 9.1 yes 25 75.8 no 4 12.1 4

Do you consider the information, published in RosStat official sources

to be up-to-date? hard to say 4 12.1 yes 28 84.8 no 1 3.0 5

Does your organization trust the RosStat official statistical

information? hard to say 4 12.1 yes 25 75.8 no 5 15.2 6

Does the quality of RosStat official publications match the modern requirements of the polygraph

printing? hard to say 3 9.1 satisfy 1 3.0

Does not satisfy 10 30.3 7

Assessment of the comprehensiveness of official

statistical information on RosStat Internet site Satisfy insufficiently 22 66.7

satisfied 13 39.4 Not satisfied 5 15.2 8

Assessment of ease of information retrieval at RosStat

Internet site. Satisfied insufficiently 15 45.5

10 Thirty-six (36) institutions were surveyed with 33 responses received.

41

Questionnaire In its officially published statistical materials RosStat aims at fully satisfaction of the needs of all categories of users in statistical information, including the government bodies in first turn. RosStat is kindly requesting you to provide the feedback to the questions bellow by marking with a tick the answer you consider to be the right one. 1. Please indicate the way you prefer to receive the official statistical information from RosStat?

Hard copies (books, collections) WEB-access Electronic versions (diskettes, CD-ROMs, E-mail)

2. Are you satisfied with the scope of information being published in RosStat official statistical sources? Yes No Hard to say

3. Are you satisfied with the timeliness of RosStat’s provision of official statistical information? Yes No Hard to say

4. Do you consider the information, published in RosStat official sources to be up-to-date? Yes No Hard to say

5. Does your organization trust the RosStat official statistical information? Yes No Hard to say

6. Does the quality of RosStat official publications (collections, bulletins, magazines, reports and etc.) match the modern requirements of the polygraph printing?

Yes No Hard to say

7. Could you please provide the assessment of the official statistical information, which is placed on RosStat Internet site, regarding its comprehensiveness.

satisfy does not satisfy satisfy insufficiently

8. Could you please provide the assessment of information retrieval convenience at RosStat Internet site. satisfy does not satisfy satisfy insufficiently

9. Could you please provide your thoughts and wishes aimed to improve the dissemination of statistical information from RosStat or any other comments. Thank you very much

42

Annex 5. Stakeholder Workshop Report and Results

(a) Seminar for Users of Statistical Information, in Moscow (February 2006)

Institutions represented: RBC-Soft University Information System, State Moscow University Research Center Institute for Statistical Research &Economy of Knowledge, High School of Economy Institute for Forecasting of the Russian Academy of Sciences Vedomosti Center for Fiscal Policy Bureau of Economic Analysis Foundation (researchers) Independent Newspaper Institute for Economy in Transition Troika Dialog Gazeta RosStat RF Interfax

Workshop participants reported that, despite the persistent legacy of distrust in Russian officialdom, confidence in RosStat by users is increasing. However, they pointed to the need for RosStat to strengthen presentation of sources and publication of metadata, to publicize new analytical capabilities developed under the project to expert users and researchers; to focus on public education on statistics in order to reinforce and accelerate this growing confidence, and to routinely investigate and publicly resolve major anomalies with other data sources, particularly at the local level. In terms of presentation, participants complained that statistical data presented by RosStat have considerable deficiencies in the way the information is presented, the statistical tables are formatted. Several participants also voiced concerns regarding the lack of wider consultations and involvement of their representatives in the process of identifying the type of information to be included in statistical publications and to be placed on the website. They further noted that departmental data are not well coordinated and their outputs are not integrated in a single system. Considerable time was taken up by the discussion of two key issues in Russian statistical system at the time. The first was regarding an ongoing public controversy about the official inflation statistics, a key indicator used by the general public as well as policy makers and the business community. Participants listened to RosStat’s defense of its methodologies but were concerned that there was a publicly demonstrated reluctance to “reveal the kitchen” to the public and other users, even if it meant resolving the controversy. The second issue related to an ongoing official debate on the need for robust, disaggregated data at the local level to facilitate fiscal decentralization efforts. It was clear from the discussion that a number of non-governmental expert users were already deeply engaged in the resolution of this dilemma but also that the issue of system-wide efficiency versus local demand in terms of sample surveys would not be easily resolved.

43

(b) Field Visit to St. Petersburg: Meeting with Users on PetroStat/RosStat (February 2006) Users of official statistics in St. Petersburg have noted a significant improvement in the quality and presentation of data produced by RosStat and the regional level statistics offices. They also acknowledged that these statistics are made available on a timely and regular basis. They have also noticed substantial improvement in data compiled by PetroStat, which has the overall responsibility of collection and compilation of official statistics in St. Petersburg. They have generally good relations with staff and management of PetroStat. Through this good working relationship, they managed to develop consumer price indices (CPI) for measuring inflation in St. Petersburg.

However, they are concerned that the statistics produced by ROSTAT are only helpful to policy and decision makers at the federal level. Policy makers in the regions and local authorities find RosStat statistics not helpful for use in their region. They would like to receive statistics that are tailored to their regional and sub-regional needs. According to them, PetroStat is a branch of the RosStat and its priority is to collect and compile data that are needed by RosStat and the federal government. For this reason, the report that PetroStat gives low priority to data needed by policymakers at St. Petersburg. They would like to see improvement in the collection and publication of gender statistics.

PetroStat has collaborative arrangements with the University of St. Petersburg. PetroStat staff members are officially teaching applied statistics at the university, and the head of PetroStat is also on the Board of this University. The two agencies worked closely in the collection and analysis of recent population census work in Russia. Students from this university are allowed to participate in PetroStat’s survey activities.

On its part, PetroStat informed the ICR mission they would like to improve on the type of statistics that they produce, but they are constrained by the current federal government law/regulations on relating to regional statistics offices. Currently, staff in the regional statistics offices receive operational budget (including staff wages) from the federal government, through RosStat. Therefore, PetroStat’s annual work program is approved by RosStat. Although PetroStat can be hired by regional or municipal governments to carry out specific statistical surveys, the law requires that funds earned from such contracts be remitted to federal government (RosStat). This type of arrangements does not give incentives to PetroStat to be innovative enough to be able to assist regional governments in producing statistics that are specific to their needs.

In order to meet its specific statistical needs, the municipal government at St. Petersburg wanted to establish its own statistics units, but it was discouraged by the fact that statistics produced by such a unit would not be recognized as official, since only PetroStat’s statistics are officially recognized by the federal government.

44

Annex 6. Summary of Borrower's ICR and/or Comments on Draft ICR

STATE STATISTICS DEVELOPMENT PROJECT (STASYS)

LOAN 4468-RU

PROJECT IMPLEMENTATION OUTCOMES

А. Assessment of Project Objectives and Project Design

Project Objectives:

The main objectives of the Project set in the Loan Agreement have been the following: (1) continue institutional reforms of the central office of the Goscomstat of Russia and its local offices in pilot regions; (2) improve data collection and data processing; (3) initiate a wide-range training program in statistical subjects; (4) make appropriate adjustments in the operations of other data collecting agencies.

The initial objectives have not been revised during the Project implementation.

Project Structure:

The following are the key parts and components of the project:

Part А. Institutional Strengthening of the Goscomstat of Russia:

Procurement of goods, works, and services to:

− Improve the organizational structure, the management strategy, and the planning capacity of the Goscomstat of Russia;

− Expose the Goscomstat staff to statistical standards and methodology;

− Strengthen statistical infrastructure of the Goscomstat of Russia;

− Improve methods and procedures used by the Goscomstat of Russia for collection, processing, and dissemination of data.

Components:

(1) Improvement of organization, management strategy and perspective planning;

(2) Fostering knowledge of standards and methodology;

(3) Strengthening of statistical infrastructure;

(4) Reorganization of data collection, processing, and dissemination.

Part В. Institutional Strengthening and Technical Assistance to other Data Collecting Agencies:

Procurement of goods and services to:

− rationalize information sharing between the Borrower’s data collecting agencies, including the Ministry of Finance, the Ministry of Economy, the SCC, and the Borrower’s Central Bank;

− improve capacity to collect, analyze, and disseminate data on external trade operations, statistical data on non-banking financial institutions and natural population growth;

− improve capacity to use statistical data for the preparation of economic forecasts.

45

Components:

(5) Improvement of Public Finance Statistics and the Compilation of General Government Sector Accounts in the SNA;

(6) Introduction and Compilation of Financial Statistics;

(7) Improvement of Coverage and Quality of BoP Statistics;

(8) Improvement of the Capabilities of the Government in the Utilization of Statistical Information for Economic Monitoring, Forecast, and Policy Making;

(9) Improvement of Vital Statistics Processing.

Part С. Project Management:

Assistance in the Project management and implementation process including the improvement of organizational and technical capacity of the Project Implementation Unit (the Economic Analysis Bureau); development and establishment of financial control and accounting system; preparation of accounts and financial reporting and their audit.

Components:

(10) Establishment of a Directorate under the existing PIU (the Economic Analysis Bureau) with the purpose of Program management to support the development of the state statistical system.

Changes in the Project Structure and Financing Arrangements:

The changes in the situation occurred due to a time gap of over one year between the Project preparation completion date and the project startup date necessitated Project components adjustment to better meet the Borrower’s needs. The adjustments are the following: some initially planned activities under Component 3 (Strengthening of Statistical Infrastructure) have been moved to Components 1 and 4; Component 6 (Introduction and Compilation of Financial Statistics) has been removed; subcomponent Improvement of Collection and Dissemination of Customs Statistics has been moved from Component 7 to form a new Component 10. Project Subcomponents have been rationalized and streamlined. Some of the initial vaguely defined subcomponents have been further elaborated. However, the above changes have not affected the initial Project objectives.

The World Bank and the Borrower have agreed to the following changes (Amendments to the Loan Agreement 4468-RU as of April 7, 2002; April 10, 2004; December 23, 2004; and June 7, 2006):

The initial project completion date (December 31, 2003) has been extended twice (April 30, 2005 and December 31, 2006);

Amendments have been made to Annex 4 of the Loan Agreement (Procurement and Consulting Services), namely, to contract value to allow for the application of some or other procurement procedures;

Changes made by the Government of the Russian Federation in financing arrangements for IFI-supported projects have necessitated modification of the Project funding ratio: starting December 25, 2004, the contribution of the Russian Federation raised to 80% with IBRD share reduced to 20%;

46

A number of benchmarks for Project monitoring and evaluation have increased; and

With the removal of some initially planned construction and repair works a number of changes have been made in expenditure categories to improve efficiency of using loan proceeds during the last year of the Project implementation:

Category Loan Agreement Supplement to the Loan Agreement

as of June 7, 2006 category (1) Consulting services including training

US$ 8,628,000.00 US$ 7,685,000.00

category (2) Goods US$ 18,910,000.00 US$ 20,171,287.64 category (3) Works US$ 384,000.00 US$ 97,712.36 category (4) Additional recurrent costs US$ 1,700,000.00 US$ 1,746,000.00 category (5) Lump sum Bank commitment fee

US$ 300,000.00 US$ 300,000.00

category (6) Unallocated proceeds US$ 78,000.00 US$ 0.00 TOTAL: US$ 30,000,000.00 US$ 30,000,000.00

In compliance with the joint agreements of the World Bank and the Borrower the initial Project Action Plan has been further elaborated during the Project implementation period based on the results of the World Bank’s supervision missions to achieve the Project objectives and to comply with socio-economic policy priorities of the Government of the Russian Federation.

В. Key Project Implementation Outcomes

The following government agencies have participated in the Project: the Goscomstat of Russia, the Ministry of Economic Development of Russia, the Ministry of Finance of Russia, the State Customs Committee (SCC) of Russia, the Central Bank of Russia11.

As part of the overall objectives the Project solved the following tasks: building Goscomstat’s capacity to present statistical data used for economic monitoring, forecasting, and economic policy formulation; reorganization of statistical data collection, processing, and dissemination; improvement of macroeconomic statistics (including banking statistics, public finance and BoP statistics, and social statistics); strengthening of statistical infrastructure (institutional and management / administration reforms within Goscomstat of Russia); modernization of technical equipment of the Goscomstat of Russia, the Ministry of Economic Development of Russia, the SCC of Russia. The key outcomes of the Project implementation are discussed below (by project Component and based on the Project performance benchmarks specified in the Loan Agreement).

11 In the context of restructuring of the Federal Executive based on Decree #314 of the President of the Russian Federation as of March 9, 2004 On the System and the Structure of the Federal Executive we would like to specify the names of the federal executive agencies that participated in the Project implementation: the Federal State Statistical Service (RosStat), the Ministry of Economic Development of Russia, the Ministry of Finance of Russia, the Federal Customs Service, and the Bank of Russia.

47

Part А of the Project

1. Component: Improvement of Organization, Management Strategy and Perspective Planning

Under the Component the Interagency Steering Committee has been set up to ensure Project implementation management and general supervision. The Board has been strengthened through the setting up of the relevant working groups for the key areas of the Project implementation.

The Component also covered a number of targets for the development of computer-based information system (CBIS) of the Goscomstat of Russia: specifically, the formulation of the CBIS development concept for all federal and regional offices of the Goscomstat of Russia (including 89 regional statistical committees); the design of the technical CBIS development project and testing of certain hardware/software solutions to improve the efficiency of collection, transmission, processing, and storage of statistical data.

The following are the main outcomes of the first stage of the Goscomstat CBIS Development Project implementation:

Establishing of integrated information systems; development of network and computer-based infrastructure with modern server units in the central office of the Goscomstat, the Hydro-Meteorological Center, the Research and Technical Design Institute for Statistical Information System (of the Goscomstat of Russia), and in 14 regional offices of the Goscomstat of Russia;

Furnishing of 11,073 working stations for the specialists of the Goscomstat of Russia with modern computer equipment (covering 42.7% of the total number of working stations);

Piloting of the Goscomstat CBIS support subsystems with the view of subsequent scaling up to cover all regional state statistics offices;

Establishing of HS fiber communication channel within the central office of the Hydro-Meteorological Center to enable information sharing and remote database operation for the central office staff;

Establishing of electronic mail and electronic documentation management systems in all regional state statistics offices given the licenses available at every client place;

Setting up of training facilities within the central office of the Goscomstat of Russia, the Hydro-Meteorological Center, and within six regional state statistics offices located in the central cities of federal districts;

Designing of a new web-site for the Goscomstat of Russia and a model site for its regional offices; development and implementation of GIS technologies for statistical data distribution;

Commissioning of a unified system for collecting, processing, storage, and presentation of statistical data.

In accordance with the Procurement Plan the Component has examined the Long-Term State Statistics Development Program; has made recommendations to prepare a program for further development of the state statistics, and has built capacity to manage HR processes within the Goscomstat of Russia.

The profound reform and development of statistics required establishing an HR information management system within the Goscomstat. To this end, training was provided for HR management staff of the Goscomstat of Russia to handle new equipment and software.

48

A new specialized web-based micro-data dissemination system with user interface has been set up to help analysts make additional analysis of household survey findings independently.

2. Component: Fostering Knowledge of Standards and Methodology

The Component involved out-of-country training for the Goscomstat staff in internationally accepted methodologies for official statistical surveys; training in Moscow and in other regions of Russia in statistical subjects and software handling, as well as training in information technology application (both in Moscow and other regions of Russia, and abroad). In addition, the Project Component established computer-based training facilities within the central and regional offices of the Goscomstat of Russia.

The trainings under the project were provided for over 3,000 staff of the state statistics of which: 370 specialists were trained abroad in international methodologies applied for official statistical surveys; 2,496 specialists were covered by in-country trainings in statistical subjects and in software handling (including distant learning); 477 specialists were trained in IT adopted to meet the needs of Russian statistics. The out-of-country trainings were supported by seven national statistical agencies. The in-country trainings were organized both in Moscow and in seven regional training facilities.

3. Component: Strengthening Statistical Infrastructure

The following are the outcomes of the Component implementation:

a) The federal draft law On Official Statistical Accounts and State Statistics in the Russian Federation was developed and has now been submitted by the Government to Parliament and has had its first reading.

b) Improvement of the calculation methodology for SNA indicators through adopting internationally harmonized standard classifications (All-Russian Classifier of Economic Activities, Classification of Individual Consumption by Purpose, etc.).

The project performed a number of scientific research and technological design activities allowing for significant improvement of SNA indicator calculation methodologies and practices.

Based on the general government sector data compilation methodology developed under the project, the 2005 nationwide statistics for general government sector revenue generation, and revenue use accounts were prepared for production. The indicators were integrated in the 2005 GDP, with subsequent regular revision of those made in January, 2007.

Currently, the above mentioned methodology is being used by regional statistical offices for general government sector indicators within gross regional product (see publication of statistics, March, 2007).

In August 2007, all 2005 general government sector accounts data will be published.

The pilot 2002-2004 calculations were made for Household End Consumption Expenditures indicator based on the Classifier of Individual Household Consumption by purpose.

The methodology adoption will be initiated after finalizing of the 2005 input-output tables.

As part of the activities to improve the compilation of input-output tables through developing standard classifications, a new methodology have been elaborated for the compilation of input-output tables using new OKVED-based data along with the conversion table for commodity groups integrated in the Foreign Economic Activity Commodity Classification (Russia) and All-Russian Classifier of Types of Economic Activity (OKVED). This enabled compilation of the 2004 imports/exports data by commodity group and tabulation of the data on resources and

49

consumption of goods and services based on All-Russian Classifier of Types of Economic Activity for the first time in Russia.

The 2004 pilot tables for resources and consumption of goods and services in Russia will be published in August, 2007.

The commodity classification developed by consultants as part of the activities to adopt commodity flow method in Russian statistics is applied for compilation of production accounts data on the federal and regional levels. The new methodological approaches developed under the project are applied for the compilation of tables on resources and goods and services consumption as well as for reconciliation of data on produced and consumed GDP. The approaches can be fully implemented following the adoption of new All-Russian Products Classifier.

The methodology proposed as part of the activities to improve the calculation techniques for GDP variables based on the tables of resources and consumption of goods and services allowed for improving calculation techniques for annual GDP data and its components.

During the first implementation stage for the two-level integrated data preparation system used for the compilation of national accounts the consultants developed a software to generate (on the federal level) interactive information resources on non-financial economic sector setup, which facilitates compilation of production and revenue generation data. The 2006 software tests helped analyze information resources used for GDP computation and revealed substantial weaknesses in terms of baseline information quality. Currently, a number of activities are under preparation to integrate this software in statistics practice and to improve the quality of baseline information used for macroeconomic calculations and thus enhance the reliability of GDP measures.

The consultative services provided under the Project were aimed to further strengthen SNA in compliance with the SNA–93 requirements and to develop measurement methodology for current market value and fixed capital use based on running inventory method, to develop methodological approaches for measurement of individual non-financial economic assets as part of the national wealth as well as to shape methodology for construction output data compilation based on integrated work items, which eventually resulted in substantial improvement of methodological framework for regular statistical survey of fixed assets, fixed investments, and construction.

c) New methodologies for different statistical areas of focus were integrated in the current statistical practices.

Tourism statistics: consultative services were provided for methodological and information support to compile tourism satellite accounts in Russia and for methodological approaches to implement recommendations of the World Tourism Organization (WTO) for Russian tourism statistics.

Trade and services statistics: statistical tools were updated for federal national statistical observation form #8 for international services Foreign Economic Relations (services) - Data on Exports (Imports) of Services in Foreign Economic Activity.

Measurements were made of the actual volume of commodities imported for domestic sales by individuals in the terms of customs clearance preferences. A number of expert analysis techniques and coefficients used by the Bank of Russia for foreign trade adjustments were revised and updated.

General recommendations were provided to improve retail sales statistics.

Agricultural statistics: a number of activities were performed for All-Russian Agricultural Census Operations based on the UNFAO program.

50

Consultants’ recommendations were implemented in the formulation of regulatory, methodological, and institutional framework for the 2006 All-Russian Agricultural Census operations.

Enterprise statistics: questionnaires and methodology were drafted for a sample statistical survey to observe the activities of individual entrepreneurs; new software was prepared.

In 2007, a pilot sample survey of individual entrepreneurs is planned for all kinds of economic activities in a number of regions of the Russian Federation.

Starting 2008, scaling up of regular sample surveys is planned to make statistical observation of individual entrepreneurs nationwide. This will help update indicators for this very segment of the economy and improve the quality of statistical data.

Labor statistics: consultative services have been provided for the following:

- The development of methodology, software, and technical framework for the compilation of data on economic activities of the population by type of household based on the population employment survey findings.

A number of activities have been performed to develop methodology, software, and hardware framework for compilation of data (as part of the micro-data base) on economic activity of the population by type of household based on the population employment survey findings.

- Development of analysis techniques for occupation/qualification labor force structure based on the recommendations of the International Labor Organization.

Standard combinatorial analysis techniques have been elaborated for the occupation/qualification structure of employed population by activity and by area of professional training based on the population employment survey data.

The following deliverables were prepared based on the 2003-2005 population employment survey: the description of additional variables and compilation algorithm for micro-data set; model interval charts for combinatorial analysis; and the analysis of occupation/qualification structure of employed population by activity and area of professional training.

d) Statistical database has been established to assess the national economy potential for innovation.

Innovation statistics: consulting services for the Improvement of Statistical Database have been provided under the project to assess innovation potential of the national economy.

In 2006, a number of recommendations made based on the international best practices to improve methodological approaches to statistical survey of innovation activities in the market economy in Russia were implemented in the review of statistical instruments used in innovation statistics. Statistical survey form #4-innovation Enterprise Innovation Activity Data, the form completion procedure, and the data compilation patterns have been amended as appropriate. This will allow for collecting the 2006 data based on the expanded list of the following: the innovation types surveyed, the sources of innovation funding, the updated methods to protect intellectual property and improve its competitiveness, and the target markets for products.

4. Component: Reorganization of Compilation, Processing, and Dissemination of Data

The following results have been achieved under this Component:

51

Statistical infrastructure have been established to allow for a wide use of sample surveys in statistics; appropriate organizational arrangements were provided to replace regular economic/business censuses with sample surveys;

Integrated census framework (including economic and agricultural censuses) complemented by sample surveys has been developed;

A number of activities were performed to develop methodology for censuses and sample surveys of individual entrepreneurs including the design of information campaign to support surveys of individual entrepreneurs;

Statistical indicators have been developed for budget expenditure efficiency monitoring;

Actions have been taken to restructure and modernize the data dissemination system, to improve survey data processing (including new software for data input, processing, and analysis), and to establish electronic document management system;

A number of hardware and software items have been procured to furnish the central office and regional offices of the Goscomstat of Russia with the aim of modernization of survey data processing, of establishing electronic mail and document management systems;

Arrangements have been made to ensure public access to household budget survey micro-data;

The Goscomstat’s data dissemination system has been restructured and modernized.

Part B of the Project

Part В of the Project was focused on institutional strengthening and technical assistance to the Goscomstat of Russia and to other agencies involved in the STASYS Project (the Ministry of Finance of Russia, the Ministry of Economic Development of Russia, and the SCC of Russia).

5. Component: Improvement of Public Finance Statistics

The Component has provided assistance to the Ministry of Finance of Russia in staff training and studying public finance statistics development best practices: Federal Treasury specialists were trained at the Central Statistical Offices of Norway and Dane, and at the Ministry of Finance of Australia.

Consulting services have been provided to support establishing of the Unified Chart of Accounts for general government sector accounting (including practical guidelines); specialists of regional finance departments have been trained in new accounts policies.

7. Component: Improvement of Coverage and Quality of BoP Statistics

The Component has been implemented with participation of the Central Bank based primarily on consulting services aimed to improve merchandise trade statistics coverage and data collection on imports of services. The following are the major outcomes of the Component implementation:

Development of organizational/methodological approaches to improve statistical data base on external trade;

Improvement of data collection on international services (imports of services) through sample surveys;

Development of methodology and measurement of index specifying the difference between CIF and FOB prices in imports of goods to the Russian Federation to improve the BoP statistics.

52

8. Component: Improvement of the Capabilities of the Government in Utilization of Statistical Information for Economic Monitoring, Forecasting and Policy Making

The Component has been implemented at the request and with participation, of the Ministry of Economic Development of Russia. The following are the outcomes of its implementation:

The central statistical databank has been established within the Ministry of Economic Development of Russia based on the Ministry’s integrated information system;

All-Russian Classification of Products by economic activity has been adopted to facilitate the integration of government information systems and resources, interagency information sharing, unequivocal identification of goods and services, statistical comparison of production, sales, exports, and imports of products (nationally and internationally);

Econometric models of the national economy of Russia have been developed for short-term and medium-term projections of the key macroeconomic indicators to serve as a basis for formulation of systemic and coherent macroeconomic policy; new instruments have been designed for analysis and forecasting of socio-economic development of Russia (both national economy and sector aggregates); methodological, modeling, and information framework has been developed for short-term and medium-term analysis and forecasting;

New computer hardware and software items have been procured for economic forecasting;

In-country and out-of-country trainings have been provided for the staff of the Ministry of Economic Development of Russia in macroeconomic modeling and socio-economic projections including forecasting tools and operational methods.

9. Component: Improvement of Demographic/Vital Statistics

This Component has been designed to provide trainings and expert advice for the Goscomstat specialists. The key outcomes are the following:

The 2002 All-Russian Population Census findings have been analyzed and methodological guidelines to improve population statistics have been prepared;

The papers of international symposium on All-Russian Population Census (2002) have been translated and their Russian-English edition have been published;

Recommendations have been provided to expand production of statistical data on causes of death;

Practical guidelines on demographic/vital statistics have been prepared.

10. Component: Improvement of Health Statistics

This Component has established the Population Health Status, Health Care Scope and Quality

Monitoring System. The process included the following:

• Establishing a databank comprising health statistical data and health-related legal and regulatory framework;

• Development of health indicators based on comprehensive analysis of health status, morbidity rates and other data collected regularly by health and social protection authorities, by the Goscomstat of Russia, and other agencies;

• Development of guidelines to improve population health status data collection.

53

11. Component: Improvement of Customs-related Data Collection and Dissemination

The Component has been implemented to assist the SCC of Russia.

The key outcome of the Component implementation has been the design and adoption of a system enabling on-line access to customs-related statistics through the procurement of hardware and SCC personnel training.

Part С of the Project

12. Component: Project Management – Bureau of Economic Analysis

The comprehensive nature of the Project requiring multi-agency involvement necessitated setting up the Project Implementation Unit for which the Bureau of Economic Analysis has been appointed on competitive basis.

The PIU performance has been evaluated as satisfactory throughout the project implementation. The Project has been successfully completed. The auditors have produced positive audit reports (covering seven audits conducted throughout the Project implementation) without any substantial reservations/qualifications.

The successful implementation of the objectives set for all project areas and components is the result of efficient PIU operations.

C. Key Impacts on the Project Implementation and Project Outcomes

In the late 1990s there was an urgent need in reforming of the Russian state statistics, namely: restructuring of data collection, processing, and dissemination framework to reflect market-based socio-economic development of Russia and to better meet the needs of statistical data users. The implementation of the STASYS Project has become a key trigger for the most significant institutional and conceptual reforms occurred in the Russian statistics over last years.

It should be noted that the Project was implemented at a time when economic and administrative reforms in the Russian Federation were underway. The changes in the country’s current situation necessitated prompt adjustment of the initially designed components and subcomponents of the Project and setting up of coordination arrangements for the Project implementation and procurement.

Among the major issues faced during the Project implementation the following should be specified:

Issues related to organizational arrangements. Involvement of five government agencies and multiple departments/units of the Goscomstat of Russia in the Project implementation required proper interagency coordination arrangements. The multi-agency involvement complicated harmonization/coordination process which led to delays in the project operations (compared to the projects with only one agency involved) on the initial stages of the Project implementation.

Procurement issues. Initially, it took some time for the Project PIU and the Interagency Steering Committee to adopt appropriate procurement arrangements in compliance with the IBRD policies and procedures taking account of specifics of the Project structure and its targets. Non-involvement of business entities capable of providing required services caused substantial delays in the Project implementation. The issue was addressed through making appropriate amendments to the Loan Agreement to specify the lots (works), for which consulting services could be provided only by public institutions.

54

Co-financing issues. Since 2002, the Project co-financing arrangements changed to channel funds through the Goscomstat of Russia based on the requests prepared by the PIU. This required amendments to the Project Implementation Agreement between the Goscomstat of Russia, the Ministry of Finance of Russia and the Bureau of Economic Analysis. Delays in coordination of the above document and lack of Government co-financing funds at that period entailed delays in signing contracts for procurement of computer equipment for the SCC of Russia and the Ministry of Economic Development of Russia. This was also one of the reasons for Project implementation delays.

The PIU was prompt to address the issues faced during the Project implementation mostly owing the close collaboration with the Goscomstat of Russia, the Ministry of Finance of Russia, the Ministry of Economic Development of Russia, the Central Bank, the SCC of Russia, and the World Bank.

Overall, the STASYS Project management system can be evaluated as satisfactory and as the one to have enabled the implementation of its objectives and targets.

D. Outcomes Sustainability

The implementation of the key STASYS Project activities has facilitated creating favorable environment for the successful operation of the country’s state statistical system under the new conditions of market economy. Overall, the Project has established the key elements of the new statistics in the Russian Federation to date, which are in line with socio-economic situation of the country undergoing transition to the market economy and with the Basic Principles of Official Statistics approved by the UN Statistical Commission.

Consulting service have been embodied in the regulatory framework of the Russian Federation, in methodological, analytical, and other deliverables used by federal executive agencies of the Russian Federation.

Under the Project activities Federal Law #108-FZ On All-Russian Agricultural Census (as of July 21, 2005) has been formulated and subsequently approved by the State Duma of the Russian Federation.

In addition, Resolution №651 of the Government of the Russian Federation On All-Russian Agricultural Census Operations (as of October 31, 2005) has been formulated.

The following methodological documents have been prepared and approved as a result of the Project activities:

1. Methodological guidelines for statistical observation of foreign investment flow in accordance with the BoP guidelines; statistical observation framework for foreign capital flow have been amended accordingly.

Stating with preparation of the 2002 first quarter report, foreign capital flow statistical data have been collected and processed based on the methodological assignment fulfilled.

2. Methodological guidelines to fix a set of consumer goods and services for cross-regional comparison of the population purchasing power.

Computation procedure for the cost of the above set is currently applied in statistics; since January, 2002, the data have been available in print in the Goscomstat publications.

3. Methodological guidelines for the replacement of goods to produce consumer price index estimates in the context of continuous product innovation.

55

The guidelines have been distributed among regional state statistical offices and applied by statistical agencies since May, 2002.

4. Methodological guidelines for basic organizations sampling to facilitate statistical observation of manufacturing prices for industrial products.

The guidelines have been distributed among regional state statistics offices starting January, 2005.

5. Methodological guidelines for calculating the cost of living index.

The activities to adopt the cost-of-living calculation will continue. The activities will include elaboration of calculation algorithms, their piloting and testing.

6. Methodological guidelines to measure informal sector employment indicators.

Quarterly preparation of the key indicators has been ensured based on the population employment

surveys (labor market survey) reflecting the employment rates in the informal sector of the

economy (both nationally and regionally). Starting with the preparation of the 2001 data, the

indicators have been published in the thematic updates based on the survey findings and have

been incorporated in the report Socio-Economic Situation in Russia prepared by the Goscomstat

of Russia.

Starting with the preparation of statistical reports for April, 2002, the methodological guidelines

have been distributed among regional state statistical offices for practical use.

7. Methodology for the preparation of labor cost indicators based on the international labor accounts best practices.

The methodology has been adopted for the calculation of labor cost aggregates for all activity types (nationally and regionally) starting with the 2003 data preparation. Aggregate labor cost indicator is a Component for calculation of labor productivity index on macroeconomic and individual economic activity levels.

8. Drafting a framework for economic census in the Russian Federation.

9. All-Russian Economic Census: census forms testing.

10. Preparation of schedule for replacing enterprise censuses with sample surveys.

The following deliverables have been provided to support the preparation of the 2006 All-Russian Agricultural Census:

- Key methodological and organizational guidelines for the preparation of the 2006 All-Russian Agricultural Census.

- Methodological guidelines for stock-taking of units to be covered by the 2006 All-Russian Agricultural Census.

56

- Census documentation for the 2006 All-Russian Agricultural Census.

- Procedure guidelines for completing questionnaires (by unit type) under the 2006 All-Russian Agricultural Census.

- Statistical tools for the 2006 All-Russian Agricultural Census.

- Training procedures for those to be involved in collecting data on the units to be covered by the 2006 All-Russian Agricultural Census.

- Training program for those to be involved in collecting data on the units to be covered by the 2006 All-Russian Agricultural Census.

Procurement and delivery of computer hardware and software has improved technological level of state statistics regional offices.

The following items have been procured and delivered for state statistics under the project: 471 servers; 11,073 working stations; 240 minicomputers; 103 data storage systems; 368 units of other telecommunication equipment; 1,630 printers; equipment for training facilities set up at the central office of the Goscomstat, the Hydro-Meteorological Center, and at six regional offices located in the central cities of federal districts.

Trainings provided for the state statistics staff have improved their professional level, has facilitated acquiring of knowledge in a number of statistical subjects and new methodologies, as well as in advanced software technologies.

Under the STASYS Project training was planned for the staff of the central office and regional offices of the Goscomstat of Russia to ensure high professional level of the staff. Successful development of the state statistics depends directly on the professional level of its managers and other specialists, on continuous enhancement of their scientific and practical knowledge and skills.

In 2001, a total of 138 specialists of the Goscomstat of Russia (of which 81 represented regional offices) improved their skills.

The trainings covered the following 11 areas of focus: use of economic census findings, administrative registers, data bases and other similar sources of information to enhance accuracy and reliability of sample survey findings; economic census operations and publication of findings; business register development and its statistical uses; methods of wage and man-power cost statistical observation; labor force survey; methodology for compilation of assets and liabilities balance sheet; methodology to measure industrial and construction products cost, and quantum indexes for specified sectors; training of specialists in methodology supporting investment measurements and quantum indexes calculation; improvement of retail sales statistical observation for non-food commodity markets, combined markets and food-commodity markets; methodology for the preparation of food balances for the EU countries; official statistics and economic system; strategy and management skills.

In 2002, a total of 93 specialists of the state statistics of Russia (of which 62 represented regional offices) were trained abroad under the STASYS project.

Skill enhancement courses covered the following relevant topics:

- Official statistics and Economic System;

- Agricultural Census Operation based on the UNFAO Recommendations;

57

- Network Information Technologies for Collection, Processing and Presentation of Statistical Data;

- Project Management for Establishing and Development of Information Processing Systems;

- Organizational and Operational Arrangements for Economic Censuses and Publication of Their Findings.

In 2004, in parallel with establishing of computer network infrastructure specialists of pilot projects were trained in Moscow in administration and operation of modernized information processing system of the Goscomstat of Russia. A total of 342 specialists from all regional offices of RosStat were trained to manage electronic mail and electronic documents.

To train specialists of the Goscomstat of Russia in applying advanced information technologies within the framework of unified system of collecting, processing, and storage of statistical data the following training courses were conducted for a total of 298 specialists of the central and regional state statistical offices in 2006: web-sphere application server; operation techniques for data user economists on the national and regional levels within the framework of the unified system of collecting, processing, storage, and presentation of statistical data; electronic digital signature management techniques.

Experts of the European Academy of Economics, Management and Law conducted advanced training for 180 employees of regional Goscomstat offices in Moscow on the following topics: Household Budget Survey Framework and Data Analysis; Interviewers Recruitment Procedure, Interview Techniques and Field Work Quality Control Training.

In addition, the Institute of International Business Law and Practice conducted training workshops for 279 specialists of the central and regional offices of the Goscomstat of Russia in Moscow, Kazan, Nizhny Novgorod, Novosibirsk, Omsk, Rostov-on-Don covering the following topics: Regional Statistics (organization, methodology and data dissemination); Use of Minicomputers for Collecting Consumer Price Data; Business Register Development and Its Statistical Uses; Compilation of Input-Output Tables based on Commodity Flow Method.

In 2003, training was conducted in Moscow, Barnaul, Nizhny Novgorod, Rostov-on-Don, and Khabarovsk for a total of 324 Goscomstat staff (of which 309 represented regional offices) at the LANIT Networking Academy Training Center on the following topics:

- Distributed Information Resource Management within Corporate Networks;

- Framework for Establishing and Operation of Computerized Information Processing Systems. Information Resource Security;

- Network Technologies and Microsoft Windows Operational Systems;

- Protection of Information Resources.

Experts of the European Academy of Economics, Management and Law provided training for 136 specialists in Moscow on the following topics: SPSS Introductory Course; Household Survey: Framework and Data Analysis; Russian Household Budget Survey Data Base.

In addition, the Institute of International Business Law and Practice proceeded with training workshops in Moscow, Kazan, Nizhny Novgorod, Novosibirsk, Omsk, Rostov-on-Don. A total of 476 specialists (of which 445 represented regional statistical offices) were trained. The following are the topics covered:

- Population Employment Survey: Labor Force Survey, Data Entry, Analysis and Publication of Findings;

- Regional Statistics (organization, methodology and dissemination of data);

58

- Enterprise Register Development and Its Statistical Uses;

- Compilation of Input-Output Tables based on Commodity Flow Method;

- Use of Minicomputers to Collect Data on Consumer Prices;

- Long-term Strategic Planning Methods.

In 2004, advanced training for specialists of the central office and regional offices of the state statistics under the STASYS Project was conducted based on distant and traditional training technologies.

Thus, a total of 1,345 staff of the regional offices of the state statistics of Russia were covered by distant training program of the International Interuniversity Education Consulting Agency: Price Survey: Organization, Processing and Publication of Data; Wage and Labor Cost Statistical Observation Methods (261 people); Small Business Survey: Theory and Practice (334 people); Network Technologies and the 2000 Microsoft Windows Operation System (433 people).

The Public Innovation Academy and IT Company conducted training in Moscow for 112 public servants of the state statistics (of which 110 represented regional offices) on the topic Automated Human Resource Management System.

The training center Networking Academy and Computer-based Information System company provided training in Moscow and in the central cities of federal districts for a total of 406 specialists of the state statistics (of which 404 represented regional offices) on the topic Hardware and Software Operation (including protection of confidential information) to support ‘One-window’ technology.

The company Craftway Corporation PLC conducted training in Moscow and in the central cities of federal districts for a total of 355 specialists of the state statistics (of which 340 represented regional Goscomstat offices) in a number of topics related to the development of information technologies.

In 2005, the STASYS Project proceeded with the distant training for the state statistics specialists. Thus, a total of 351 specialists of regional offices were trained by the International Consulting Interuniversity Agency on the topic Statistical Register Operation based on ‘One-Window’ Interagency Information Coordination Mechanism.

The SPSS training center provided training for 194 people (of which 170 represented regional statistical offices) on the topic Application of SPSS Basic Software.

In 2006, the International Consulting Interuniversity Agency jointly with Bonn Academy conducted training for 74 management staff of the regional offices and six specialists of the central state statistics office on the topic Statistical Operations and Management.

During the training the specialists were exposed to the international state statistics best practices (namely advanced statistical operations and management) and to the current national and international statistics management system to apply the acquired knowledge for better management within regional state statistics offices in Russia.

In addition, a total of 352 specialists of regional state statistics offices were covered by the advanced course provided by the Eurasian Open Institute on the issues related to operations technology for federal level economists using the unified system of collecting, processing, storage and presentation of statistical data.

The STASYS project has set up training facilities within the Goscomstat of Russia including 7 classrooms set up in the relevant regional state statistics offices located in the central cities of federal districts (Petersburgcomstat, Rostovsky Oblstat, Nizhegorodsky Oblcomstat, Sverdlosky

59

Oblcomstat, Novosibirsky Oblcomstat, and Khabarovsky Kraicomstat), and within Hydro Meteorological Center of the Goscomstat of Russia.

The federal level training facilities include: the directorate of studies; a specialized multi-functional lecture room; a specialized operations/training unit to prepare documentation for multi-media cessions; a server system module for distant learning; operations and training oriented technological tools with standard number of working stations; security and video surveillance system.

The regional training facilities include: a specialized multi-functional lecture room; a server system module for distant learning; operations and training oriented technological tools with standard number of working stations (10).

The training facilities within the Goscomstat of Russia and regional level training facilities are used to train specialists of the central and regional offices of the state statistics (based both on traditional and distant training technologies).

E. Performance of the World Bank and the Borrower

1. World Bank Lending

The World Bank performance for Project planning and supervision can be evaluated as satisfactory. The Bank promptly responded to the changes caused by changing socio-economic situation and policies; this, in its turn, facilitated more efficient use of Project funds and helped avoid changes in the key Project objectives. Close Project implementation supervision on the part of the Bank Project team ensured the execution of the Loan Agreement and compliance with Bank Procurement guidelines. The collaboration with the World Bank Moscow office was fruitful and helped address emerging issues and thus contributed to successful project implementation.

2. Borrower / Project Implementation Unit

To ensure the STASYS Project implementation supervision and prompt/efficient coordination of Project management issues the Inter-Agency Steering Committee comprised of participating agencies has been established. The Committee has been strengthened by six working groups set up for the key project components. The members of the Working Groups met as required to address the current project issues.

All financial and organizational decisions under the Project were made by the Goscomstat of Russia jointly with the Bureau of Economic Analysis in consultation with other relevant agencies involved in the Project with consequent approval by the Interagency Steering Committee.

To ensure proper project management on the part of the Bureau of Economic Analysis the Project Implementation Operational Manual has been developed including key guidelines for day-to-day project operations: policies and procedures for project management, procurement, financial management, HR management, etc. The internal financial management information system established under the project has facilitated efficient project implementation supervision including procurement plan implementation and efficient accounting. Throughout the Project implementation the database has been regularly updated for consulting firms, commodity and software suppliers, and individual consultants who participated in the competitive bidding under the Project.

The Bureau of Economic Analysis prepared all required project reporting content-driven documents and financial data on time and submitted them to the Goscomstat of Russia, the Ministry of Finance of Russia, the Ministry of Economic Development of Russia, and to the

60

World Bank. This ensured efficient Project implementation monitoring on the part of all parties involved.

F. Conclusion (Lessons Learned)

A total amount of USD 29,942,243 out of USD 30,000,000 of 4468-RU Loan proceeds has been disbursed during the STASYS Project implementation.

The Project has become one of the key activities to initiate institutional and methodological reforms in Russia’s statistics. The large-scale reforms implemented under the Project have covered the activities of the key executive agencies of the Russian Federation: the Federal State Statistics Service, the Ministry of Economic Development of Russia, the Ministry of Finance of Russia, the Federal Customs Service, and the Bank of Russia.

The Project has established new key elements of the state statistics in the Russian Federation consistent with the Basic State Official Statistics Principles approved by the UN Statistical Commission. Alongside with that, actions are being taken to further improve state statistics, which is especially important in the light of Russia’s OECD accession and strengthening cooperation with the European Union as well as with IBRD and IMF. In this context compliance with international standards in statistics contributes much to the consolidation of Russia’s position.

In order to implement and successfully complete the systemic reform process for the state statistics of the country, to bring it into line with international statistical standards, to build on the results achieved, and to further develop statistical system within the framework of economic policies of the government and acceleration of computerization process there is still a need in addressing a number of pending major methodological and operational issues covering the following areas:

– Modernization of statistical methodology through establishing the System of National Accounts and adopting international economic and statistical classifications;

– Development of information technologies for collection, compilation, storage, and dissemination of data and integrated technology for forecasting and analysis;

– Improvement of labor, demographic/vital, and social statistics;

– Build human resource capacity in statistics.

To sustain the results achieved to date and to continue the state statistics modernization process in the Russian Federation based on international standards in methodology and technology of collection, compilation, and dissemination of statistical data the implementation of the Second State Statistics Development Project is planned. The Project preparation is now completed and its content is in line with the priority areas of collaboration of the Russian Federation with international financial and economic institutions.

61

Annex 7. Comments of Cofinanciers and Other Partners/Stakeholders (a) Borrower/implementing agencies Ministry of Finance of the Russian Federation in its letter as of December 19, 2007 #04-04-09/9090 to the Country Director for Russia advised that it had no comments or suggestions regarding the ICR and performance of the State Statistical System Project. The Rosstat in its letter as of November 12, 2007 #ВС-05-38/3169 informed the World Bank Moscow office it had no comments on ICR. (b) Co-financiers In consultations with the authorities from Japan Embassy in Russia, it was indicated that they found the report in general acceptable. All comments on the PHRD grant contributions were adequately incorporated into the report. Subsequent communications in writing are filed in the project folder in IRIS.

62

Annex 8. List of Supporting Documents 1. Minutes of Project Concept Review 2. Aide Memoires, Preparation Missions 3. Aide Memoires, Supervision Missions 4. Agreed Minutes of Negotiations 5. Project Appraisal Document 6. Project Information Document 7. Credit Agreement 8. Consultant Reports 9. Project and Implementation Status Reports 10. Letter from Federal State Statistical Service of the Russian Federation (Rosstat) 11. Letter from Ministry of Finance of the Russian Federation