module monitoring and evaluation of labor programs

MO

DU

LE 7

OVERVIEWThe specific objectives for monitoring and evalua-tion of labor programs are to:

• Assess financial and economic returns.Labor programs involve spending consider-able resources in the short run to reap somegain in the longer term. Consequently thedecision to undertake a labor restructuringprogram should consider the financial andeconomic returns involved, much the sameas would an investment decision.

• Learn from past experiences. Monitoringand evaluating a program’s effectiveness canhelp assess what works and what doesn’t,and the lessons from past experiences caninform future labor programs.

• Reduce costs to government. Most laborprograms can be very costly. Cost-benefitanalyses help avoid the experience in onecountry where a policy of generous levels ofseverance pay (the highest in the region) had

been set, but where the key decisionmakerlater admitted, “We didn’t actually work outhow much it was going to cost us. We justmade our decision after looking at other[voluntary departure] schemes elsewhere,but felt we should increase it somewhat.”

• Help make the case for work force restruc-turing. There will always be opponents ofwork force restructuring. Cost-benefit analy-ses can help provide key information forcommunication and negotiation (see module6).

• Assess financial sustainability and identifyrequired financial resources. Cost-benefitanalyses can help provide insight into thefinancial sustainability of the overall pro-gram by taking into account the overallcosts of the redundancy program and esti-mating the impact of possible future addi-tional redundancies. These costs should takeinto account all associated costs: compensa-tion, redeployment, and the costs for the

173

The capability toundertake effectivemonitoring, analysis,and evaluation willenhance thecredibility andreputation of theimplementingagency.

7Monitoring and Evaluation of Labor Programs

This module provides guidance on why and how to evaluate the costsand benefits of labor restructuring programs and how to set up effec-tive monitoring systems to track progress and learn from experience.

MODULE

pension system where there are early retire-ment programs.

Early analysis can be critical in helping govern-ments assess severance options and the size andscope of the resource envelope for which funding isrequired. Consider the following example: An eco-nomic reform–implementing agency had embarkedwith some success on a pilot program of privatiza-tion. In most cases work force restructuring tookplace prior to privatization (and in some cases, clo-sure). The agency developed a five-year plan foraccelerated privatization that would include majorinfrastructure enterprises. Empirical estimates oflikely levels of downsizing were made, based onexperience in the pilot state-owned enterprises(SOEs) and from similar enterprises elsewhere inthe region. Those estimates revealed that unexpect-edly high levels of expenditure on voluntary depar-ture might be required in 2005 and 2006 (aboutUS$200 million annually) when restructuring oflarge mining and infrastructure enterprises wasplanned. This analysis helped to inform planningby government (the ministry of finance, in particu-lar) on (a) the sequencing of privatization and (b)discussions with donors on a new lending facilityin support of privatization and state enterprisereform.

The audience for analyses will extend beyond theimplementing agency. Although analysis is anessential input for decisionmaking on labor pro-grams, work force restructuring is both a technicalissue and a political issue. The logic and rigor oftechnical analysis may not always be the decisivefactor, and the quality of presentation is importanttoo. In practical terms this means that:

• The findings are presented in a way that willbe accessible to policymakers and to a wideraudience (for example, an overly academicpresentation may not communicate conclu-sions effectively).

• The presentation of financial and economicanalysis will reveal political costs and bene-fits. Simple examples are:

– Comparing the number of workers whomay lose their jobs with the number of

beneficiaries of private participation ininfrastructure (PPI) and infrastructuresector reforms (for example, projectednumber of households expected toreceive new water or power connections).

– Presenting estimates of likely employ-ment outcomes for the sector as a whole,not just the state enterprise. For example,in telecommunications immediate short-term job reductions in the state enterprisemay be quickly made up by new jobs innew entrants in mobile and data commu-nications.

– Expressing the financial cost savings togovernment from work force restructur-ing in terms of the alternative social ben-efits that could be provided from thosesavings (that is, number of new ruralschools or health clinics built a year,annual maintenance of rural roads, annu-al salaries of school teachers).

• Financial and economic analysis of the laborprogram needs to be complemented bystakeholder analysis (which includes politi-cal concerns), as discussed in module 6, andcost-benefit analysis of the wider case forPPI, in situations where opponents of laboradjustment are likely to challenge PPI itself(see the negotiations section in module 6).

The usefulness of analysis is constrained by two fur-ther factors: time and the availability of the data.Governments often must make their decisions basedon limited and incomplete information—they rarelyhave the luxury of the time needed to conduct a fullanalysis and to receive robust conclusions. Evenwhen analysis is done, the quality of available datamay restrict the usefulness of analysis.

ASSESSING FINANCIALRETURNSGovernments, like the private sector, need to assessthe financial costs and benefits of a work forcerestructuring program. Unlike the private sector,however, governments also need to assess a pro-

Labor Issues in Infrastructure Reform: A Toolkit

174

Complementfinancial analysiswith stakeholderanalysis, andpresent the resultsin ways that revealsocial costs andbenefits.

MO

DU

LE 7

gram’s economic costs and benefits to society, or theeconomy, as a whole. The key questions and poten-tial tools of analysis are summarized in table 7.1.

As outlined in module 1, the immediate triggers forwork force restructuring are often financial crisis,not economic crisis. In such cases the financialanalysis can be a more critical question for govern-ment than is economic analysis.

Financial Costs—A ChecklistTable 7.2 provides a checklist of potential financialcosts that may be incurred and that must be includ-ed when assessing financial returns. Essentially, thefollowing equation captures the situation:

Financial costs = SC + RC + FC + CC + JC + TC + UC

where SC is the present value of severance costs,RC is the current net value of retirement costs, FCis the existing value of transportation for worker

and family, CC is the cost of counseling, JC is thecost of job-search assistance, TC is the cost oftraining, and UC is the present estimated value ofunemployment benefit and other social payments,

When collating those costs the implementingagency will need to ensure that:

• All costs are viewed from the perspective ofgovernment as a whole. For example, costsshould include any incremental cost to gov-ernment of additional unemployment bene-fits or pensions for displaced workers.

• Costs are all brought to present values. In amajor restructuring, downsizing is likely tobe a phased activity, so planned downsizingtwo or three years hence should be appro-priately discounted. In addition, the esti-mated costs to the pension plan of anyearly retirement benefits paid in future mayneed to be measured in terms of presentvalue.

Monitoring and Evaluation of Labor Programs

175

Work forcerestructuring oftenuses batches ofworkers over time.The costs (andbenefits) of futurebatches need to bediscounted to acommon presentvalue.

MO

DU

LE 7

Table 7.1: Financial Analysis—Key Questions and Tools

Question Tool

How much will the program cost? • Initial estimates of total program cost, based on rough assumptions of severance costs (see the quick calculator on the accompanying CD-ROM)

What are the financial benefits? (How much • Net present value of net cash flows to and from money will government save?) government, taking account of savings in:

– Salary and related costs and allowances– Retirement benefits– Other nonsalary benefits

How can the program be funded? • Financial gap between cost estimates and potential funding sources (consider budget sources, privatiza-tion revenues, commercial loans, donor funds, and so forth)

How long will the program take to pay for itself? • Payback analysis—how long before the costs of a labor program are recovered through savings in reduced wages and other labor-related costs?

What is the financial impact of different • Payback and net present value analyses for different approaches to selection and work groups of employees (grade, age group, operating force restructuring? units)

• Analysis of alternative severance formulas

Looking at the economy as a whole, do the • Substitution of financial costs and benefits with economic benefits of work force restructuring economic costs and benefits in analysesexceed its costs?

Financial Benefits—A ChecklistTable 7.3 summarizes sources of benefits from alabor program, as portrayed by the followingequation:

Financial benefits = W + R + B + O + P

where W is the present value of wage savings, R isthe current net value of retirement benefits saved, Bis the present value of savings in kind (nonwageallowances and other benefits), O is the currentvalue of reduced operating costs, and P is the esti-

mated increase in PPI transaction proceeds result-ing from the labor force adjustment.

As with costs, benefits should be assessed in thecontext of overall government spending. If anenterprise gains cost savings as a result of the trans-fer of staff to another enterprise or elsewhere ingovernment, then overall there is no cost saving togovernment (as in the example of the transfer ofAqaba rail employees to the Jordan PhosphateMines Company, described in module 4, box 4.6).The pension analysis similarly needs to consider


176

Financial benefitsshould also beconsidered in thecontext ofgovernment as awhole, not just ofthe infrastructureenterprise.

MO

DU

LE 7

Table 7.2: Checklist—Financial Costs of a Labor Program

Item Comments and examples

Direct costs of separationPresent value of severance costs (SC) • Includes all the costs described in module 5: ex

gratia, statutory payments, gratuity, bonuses, allowances per enterprise rules, payments negotiat-ed in individual or collective labor contracts.

Net present value of retirement costs (RC) • Estimated present value of future obligations to pro-vide retirement benefits (using realistic estimates of life expectancy, investment returns, and so forth)

• Costs should include possible arrears in pension contribution and any additional investments needed to ensure the financial sustainability of the pension scheme.

Present value of transportation for worker • Costs vary according to the enterprise (central or and family (FC) dispersed), country (size, transportation costs),

nature of the work force (locally hired or nationally hired).

Redeployment costsCounseling (CC) • Counseling costs based on expected take-up rates.

• Assume that almost all workers will receive counsel-ing.

Job-search assistance (JC) • Little data on actual uptake in severance programs—perhaps 60–75 percent

Training (TC) • Only a percentage of the work force is likely to undertake training—perhaps 10–30 percent, accord-ing to experience.

Other costsPresent value of estimated additional • Costs of social insurance to workers: Estimate addi-

unemployment benefit and other social tional social insurance payments according to rules. payments (UC) Estimated periods of unemployment can be derived

from social security records, public employment service records and interviews, private sector place-ment agencies, and surveys of workers displaced previously.

the whole government context. If pension costs aremerely transferred from the enterprise to anotherpublicly guaranteed scheme, then there may be nocost saving for government.

Most benefits are self-evident, but pensions can becomplex. In a defined-benefit plan with a highratio of pensioners (receiving pensions from theplan) to employees (contributing to the plan), a

large number of departures could tip the plan intofinancial insolvency. Complexity can also arise ifdifferent workers within enterprises have differentpension programs. To illustrate, if some workersare public servants and others are enterprise work-ers subject to a general labor code, net savings togovernment might be higher for retrenchment ofthe public servants because they have more gener-ous pension benefits.


177

MO

DU

LE 7

Table 7.3: Checklist—Financial Benefits of a Labor Program

Item Comments and examples

Direct benefits of separationPresent value of wage savings (W) • This is usually the largest single source of savings:

Includes wages of separated staff plus additional costs, allowances per enterprise rules, payments negotiated in individual or collective labor contracts.

• Eliminating ghost workers may be an important and immediate source of financial savings in some enter-prises.

Net present value of savings in retirement • Some pension costs must be paid even without benefits (R) restructuring, so the difference between what would

be paid with and without the labor program should be estimated.

• Benefits also arise from the elimination of ghost pen-sioners.

Present value of savings on benefits in kind— These benefits can include but are not limited to:both variable and semivariable (B) • Transportation to work, subsidized food (canteens),

and heating fuel costs• Reduced medical costs• Child support and childcare (kindergartens)• Free or subsidized housing or reduced housing

maintenance costs.

Other benefitsPresent value of reduced operating costs (O) • Reduced costs, such as transportation and vehicle

costs, reduced administration costs (fewer support staff), reduced pilferage.

• In some cases disposing of employee housing (per-haps cheap sales to workers), surplus offices, depots, and vehicles will reduce running costs.

Estimated increase in PPI transaction • Increase resulting from downsizing per se.proceeds as a result of work force • Increase (or decrease) resulting from more (less) flex-restructuring (P) ible labor contracts.

• Increase resulting from faster PPI transaction.• These may be difficult to estimate but are potentially

significant. López-de-Silanes (2002) found that a 20 percent reduction in the work force before privatiza-tion led to a 24 percent increase in net privatization price (see box 7.5).

Financial Payback AnalysisWhere the costs of work force restructuring arewholly front-loaded, the simplest method of analy-sis that could be considered is undiscounted pay-back (or breakeven) analysis. In the example pre-sented below, an initial expenditure of $100,000 inseverance payments is “paid back” by the end ofyear 4 as a result of annual savings (wages andother staff costs) of $25,000. The simple payback(or breakeven) period is calculated as the numberof years it takes for the wage bill savings to equalthe financial costs—in this case 4 years.

Work force restructuring Cash Net cash

Year costs ($) savings ($) flow ($)

0 100,000 –100,000

1 25,000 –75,000

2 25,000 –50,000

3 25,000 –25,000

4 25,000 0

5 25,000 25,000

6 25,000 50,000

The introduction of discounting in the aboveexample gives a more accurate picture of the pay-back period. By taking into account the lesservalue of money tomorrow instead of today andassuming an annual discount rate of, say, 10 per-cent, the initial expenditure of $100,000 is paidback not at the end of year 4 but during the courseof year 5.

Work force Discoun-

restruc- ted Net turing Cash Present values cashcosts savings value at flow

Year ($) ($) factor 10% ($) ($)

0 100,000 1.000 100,000 –100,000

1 25,000 0.909 22,725 –77,275

2 25,000 0.826 20,650 –56,625

3 25,000 0.751 18,775 –37,850

4 25,000 0.683 17,075 –20,775

5 25,000 0.621 15,525 –5,250

6 25,000 0.564 14,100 88,850

The payback period is calculated on the basis ofthe fraction of year 5 needed to bring net cash flowto zero. In this case:

Payback = 5 –(–5,250/25,000) = 5.21

Some Results of Payback Analyses

In a survey of work force restructuring programsin six countries, Svejnar and Terrell (1991) foundthat payback periods varied from just four monthsto 4.7 years. Haltiwanger and Singh (1999) evalu-ated the financial returns of 41 downsizing opera-tions based on World Bank internal documents fora range of civil service and state enterpriseretrenchment programs. Their evaluation includeda discounted financial payback method, whichassumed a 10 percent annual discount rate. For the24 operations with sufficient information to calcu-late payback analyses, their results can be summa-rized as follows:

• In 9 cases there were no net financial bene-fits from downsizing. These cases implythat there is never any payback (that is, thepayback period is infinite). These circum-stances arose as a result of rehires, newhires, or an increase in wages for theretained workers.

• In 15 cases there were positive financial ben-efits and the average payback period was2.3 years. Fourteen of the programs hadshort payback periods of between 0 and 3.6years—the one exception being the Bolivianmining corporation where the payback peri-od was 10 years. The other 3 enterprises inthe sample had payback periods of 3.40(Bangladesh jute), 1.44 (Pakistan publicenterprises), and 1.56 years (Argentina pub-lic enterprises). Programs with immediatepayback periods (0 years) were those thatinvolved involuntary reductions withoutdirect compensation of other assistance.

These findings are striking. On the one hand, manylabor programs appear to offer exceptionally goodrates of return. Few investment projects display such


178

The simplest cost-benefit analysis thatthe implementingagency can make isan undiscountedfinancial payback.

Most evaluations ofwork forcerestructuringprograms revealvery fast paybackperiods.

MO

DU

LE 7

high financial returns as the 15 cases mentioned.Other programs, however, have not recovered theircost, often because of the problems of adverse selec-tion and rehiring described in module 5.

Haltiwanger and Singh.

Discounted Cash Flow AnalysisWork force restructuring can be viewed as a proj-ect. Classic investment projects are based on an(often high) initial capital investment, followed bya stream of positive cash flows arising over a num-ber of years. Those positive cash flows are derivedfrom productivity improvements that lead to high-er revenues or greater savings. Work force restruc-turing shares a similar pattern, so the tools ofinvestment analysis can be applied equally well to awork force restructuring project as to a capitalinvestment project. (A sample spreadsheet is pre-sented on the CD-ROM.)

Sample spreadsheet for analysis of labor projects.

Where work force restructuring involves significantdownsizing, it is difficult to persuade some to seethis as a “productive” investment. Downsizing isproductive, however, in the sense that it removesand reallocates unproductive labor to more pro-ductive activities elsewhere. It increases labor (andoften total factor) productivity within the enter-prise. Where there is a genuine surplus within thework force, the jobs are not “real” jobs, and sur-plus workers’ marginal productivity within theenterprise is likely to be close to zero. Their expect-ed productivity will be greater in other employ-ment or other activities outside the enterprise.

When using discounted cash flow analysis tech-niques for labor adjustment, the implementingagency should check that:

• A mix of indicators is used, rather than relyingon a single measure such as internal rate ofreturn or net present value (see box 7.1).

• An appropriate discount rate is selected.Choosing the right discount rate is a com-


179

Many people wouldobject to the ideathat downsizing canbe seen as a“productive”investment, butwork forcerestructuring can beanalyzed using thenormal tools ofinvestmentappraisal.

MO

DU

LE 7

Box 7.1: Indicators for Cost-Benefit Analysis

Three main indicators for cost-benefit analy-sis are:

1. Internal rate of return (IRR) is the discountrate at which the future streams of costs andbenefits are equal. The higher the IRR, thebetter the project, so the IRR method is aconvenient way to compare different alterna-tive options in labor programs.

2. Net present value (NPV) is the differencebetween the discounted streams of futurecosts and future benefits. If costs exceedbenefits, the NPV is negative; if benefitsexceed costs, the NPV is positive. The NPV isthe value, discounted to the present, ofundertaking a work force restructuring projectrather than not doing so. NPV assessmentsrequire that a predetermined discount rate isselected. One criticism of NPV assessmentsis that, when comparing alternative restructur-ing proposals, the decision rule would selectthe largest project (giving the highest NPV)

over a smaller project with a higher IRR but alower NPV.

3. Benefit-cost ratio (BCR) is the ratio betweendiscounted total benefits and costs. Thus, ifthe discounted benefits are $150 million andthe discounted costs are $100 million, theBCR is 1.5 (and the NPV is $50 million). TheBCR is a useful check to the NPV process, asa way of spotting program options that offerattractive NPVs only because they are large.Reporting of BCRs again demands mentionof the discount rate used.

In complex activities, such as work force restruc-turing, a single number—be it IRR, NPV, orBCR—is unlikely to be enough for informed deci-sionmakers. Sensitivity analysis shows how varia-tions in the key assumptions underlying theanalysis influence the expected outcomes of therestructuring program. At its simplest, this meansrunning a spreadsheet model under differentassumptions and presenting these variations in atable.

plex issue but, in practice, sources of infor-mation for the rate are the general interestrate defined by the ministry of finance forthe application of public funds—the bestsource—or offices of international lendingagencies such as the World Bank andregional development banks.

• Costs and benefit flows have been calculatedfor a sufficient period. If discount rates areabout 8–12 percent, then a 20-year period islikely to be sufficient.

ASSESSING ECONOMICRETURNSThis section outlines the rationale for, and elementsof, economic analysis of labor programs. Theresources listed at the end of this module includeseveral examples of economic analyses of workforce restructuring in public enterprises.

RationaleAs discussed above, the near-term financial benefitfor government is sometimes the critical factor forcash-strapped ministries of finance faced with anurgent need for infrastructure enterprise reform.However, economic analysis is also needed for thefollowing purposes:

• First, it assesses impact on aggregate outputor welfare. Financial analysis tells nothingabout whether displaced workers are, in theaggregate, more or less productive followingthe labor program. It is quite possible that aproposed labor program can be attractivefrom the financial analysis perspective, butcan fail when subjected to economic analysis.

• Second, it provides an answer to opponents ofwork force restructuring in PPI, who mayargue that by ignoring the wider economiccosts and benefits, government is making a baddecision (see, for example, the South Africancase presented in box 6.9 of module 6).

• Third, it may be a requirement of interna-tional funding agencies whose lending or

funding procedures need economic as wellas financial analysis.

Labor productivity issues are central to an econom-ic view of work force restructuring, which sees theprocess as a reallocation of resources within theeconomy. From an economic cost-benefit perspec-tive, the cost of restructuring must be met by anincrease in worker productivity following displace-ment. The effect of moving workers out of the PPIenterprise into other parts of the economy can leadto negative outcomes if overall labor productivityfalls, neutral outcomes if overall labor productivityis unchanged, or positive outcomes if overall laborproductivity rises. Much therefore depends on theassumptions regarding marginal productivity of theworker in the enterprise compared with the mar-ginal productivity of his or her activities followingretrenchment.

Economic vs. Financial CostsEconomic cost-benefit analysis is similar inapproach to the discounted cash flow approach tofinancial analysis. As the spreadsheet sample on theaccompanying CD-ROM shows, both financialand economic analysis can be combined in thesame cost-benefit model. Financial costs and bene-fits are substituted by economic costs and benefitsin the economic cost-benefit analysis.

Differences between economic and financial costs(summarized in table 7.4) are as follows:

• Financial costs of severance (item 1): Ineconomies with no tax distortions or subsi-dies, and no fiscal budget constraint, thefinancial costs of severance would not nor-mally be treated as an economic cost.Rather they would be seen as a neutraltransfer payment. Most developing coun-tries, however, do have distortions andbudget constraints, so severance paymentsto workers divert public funds from otheruses. Except in a few upper-middle-incomecountries, a conservative approach would beto treat the economic costs of severance as100 percent of the financial costs.


180

Economic analysisprovides aperspective thatfinancial analysisalone cannot.

MO

DU

LE 7

• Financial costs of early retirement (item 2):Similar considerations apply for the costs ofearly retirement as for severance. In practice,the principal difficulty is likely to be the esti-mation of the present value of the net finan-cial costs of early retirement.

• Financial costs of redeployment (item 3): Ifprivate sector firms are providing redeploy-ment services (training, counseling, outplace-ment) then the full market price of the train-ing can be taken as the economic resourcecost of redeployment. Where governmentsdelegate redeployment to state agencies thatare heavily subsidized and operating belowcapacity, the economic costs are likely to bebelow financial costs, and an adjustmentfactor should be applied.

• Marginal productivity of employees in theSOE (item 4): If there is a genuine surplus ofworkers, their retrenchment will not resultin a loss of productivity in the enterprise,and the marginal productivity of employeesin the enterprise can be set at zero. This is areasonable assumption, if the work forcerestructuring strategy adequately tackles therisk of adverse selection (for example,through close targeting of workers offered

severance, through restricting downsizing tocadres with obvious levels of surplus labor,or through a mixed government–investorapproach (see table 4.1, module 4).

• Financial savings on wages (item 5): This islikely to be one of the largest sources of sav-ings. Two points are important:

1. Given the evidence of rehiring in manydownsizing programs, an explicit adjust-ment factor to take rehiring into accountmay be appropriate in the financialanalysis. If workers are hired elsewherein government, or by the same enterprise,financial saving will be reduced. Forexample, employees of the Sri LankaTransport Board were induced to leavewith severance funding, but were beingrehired almost simultaneously (Svejnarand Terrell 1991). High numbers ofworkers rehired following a labor pro-gram are an indicator of the failure of theprogram, both in a financial sense (awaste of public money in financing thecosts of the workers’ departure) and inan economic sense (a failed attempt toreallocate the workers’ labor to moreproductive use). Some new hiring is occa-


181

MO

DU

LE 7

Table 7.4: Economic vs. Financial Costs and Benefits

Include in Include in Item Cost and benefit items financial analysis? economic analysis?

Costs

1 Financial costs of severance Yes Yes—adjusted

2 Financial costs of early retirement Yes Yes—adjusted

3 Financial costs of redeployment Yes Yes—adjusted

4 Marginal productivity of employees in the SOE No Yes

Benefits

5 Financial savings on wages Yes No

6 Financial savings on nonwage benefits Yes No

7 Marginal productivity of worker outside the SOE No Yes

8 Marginal productivity value of labor savings No Yes

9 Increase in privatization proceeds from downsizing Yes/No No

10 Increase in privatization proceeds from faster PPI Yes/No No

sionally needed, however, to bring miss-ing skills to the work force.

2. A decision needs to be made on whetherto adjust wage savings to take account ofenterprise profits. All other things beingequal, a wage saving should ultimatelyreturn to the budget whether the enter-prise is fully or partially subsidized.

• Financial savings on nonwage benefits (item6): This should take into account savingsfrom reduced expenditure on all allowances,plus projected cost savings arising from theclosure or reduction of services provided toemployees such as food, medicine, housing,education, and cheap loans.

• Marginal productivity of a worker outsidethe SOE (item 7): This is the product of (a)the probability of an employee’s engage-ment in a productive activity and (b) the netincome produced by this activity. The mar-ginal productivity of the retrenched workerafter he or she has left the enterprise there-fore depends greatly on his or her circum-stances following retrenchment (see“Assessing the Effects on Workers’Welfare” in this module for a list of poten-tial postretrenchment circumstances).Incomes will depend on worker attributes(age, education, and, in some situations,gender); market conditions (overall eco-nomic growth rate, labor market supply,and demand); and worker location (capitalcity, urban, rural).

Estimates of workers’ marginal productivityshould be based on local market informa-tion. This should be assumed to be less thancurrent salaries. But what factor should beapplied? Evidence suggests that even inindustrial economies a permanent earningsreduction of 15–20 percent can be expectedamong displaced workers, but reductionsmay be even greater in developing countrieswith public sector wage premiums (see box1.1 in module 1). Moreover, many workersgo not into formal employment but intoinformal employment or self-employment,

about which there are fewer data. Estimatesof marginal productivity—disaggregated bythe major classes of workers—can be bestsourced from data of actual wage or incomelevels outside the public sector, gained fromlabor surveys, national statistics, agriculturaleconomics studies and research, householdincome surveys, interviews with placementand business support agencies, and (if possi-ble) focused follow-up studies from earlierretrenchment exercises.

• Marginal productivity value of labor savings(item 8): The economic value of the wagesavings following downsizing should reflectthe opportunity costs of these savings ifinvested by the enterprise in expanded serv-ice areas, improved productivity, orimproved quality of service by the enter-prise. Assuming the enterprise is budget con-strained, then this opportunity cost can beset as 100 percent of the financial cost.

• Increase in privatization proceeds fromdownsizing (item 9): Global evidence of theincrease in privatization proceeds as a resultof downsizing is limited. The best study(that of López-de-Silanes 1997) suggests anincrease of net privatization prices of 12 per-cent for every 10 percent reduction in thelabor force. However, because investorbehavior is difficult to predict, a conserva-tive approach would be to ignore theprospect of improved privatization prices inboth economic and financial analysis, ortreat it as a factor in a sensitivity analysis.

• Increase in privatization proceeds fromfaster PPI (item 10): A key rationale forwork force restructuring (particularlythrough voluntary departure and earlyretirement) is that it helps remove barriers toPPI investment by reducing political andworker opposition to PPI.

In the absence of country-specific estimates,a conservative approach to cost-benefitanalysis will probably ignore this time effecton privatization proceeds, not least becausethere are many potential sources of delay to


182

MO

DU

LE 7

a transaction in addition to labor issues.Nonetheless, in some circumstances the timeeffect may be a significant source of benefits.

EVALUATING LABOR MARKETPROGRAMS

Constructing the CounterfactualAnalysisEvaluation is the periodic assessment of the rele-vance, performance, efficiency, and impact of theproject in relation to stated goals. It differs frommonitoring in that it is not an essential task for theimplementing agency. Evaluation is mainly con-cerned with impact, which may only be measura-ble toward the end of implementation or in lateryears and so is often better done by a separateagency independent from implementation.

A central requirement of any evaluation is separa-tion of the effects that would have happened any-way from those that resulted from the intervention.Before-and-after comparisons alone are not suffi-cient. If earnings rise after training, for example,that may be the result not of the training but ofchanges in the macroeconomy or local changes inlabor demand or of such worker-specific attributesas life-cycle changes in earnings.

Evaluation therefore requires a counterfactual test,which is normally provided by a control or com-parison group of workers who did not participatein the severance or redeployment program. Box 7.2illustrates the importance of creating such groupsfor a hypothetical redeployment training program.

Counterfactual analysis can use either:

• Control groups, which consist of partici-pants that are selected at random within awell-defined population from which themembers of the treatment group are alsoselected

• Comparison groups, which consist of partic-ipants who are purposively matched to theparticipants in the treatment group.

Counterfactual analysis demands careful choice ofthe scenarios against which the outcomes of laborprograms are compared. For example, if the intro-duction of a PPI project in a declining public sectorport leads to the loss of 1,000 cargo-handling jobsout of a total of 3,000 jobs through voluntarydeparture over three years, which of the followingis the appropriate counterfactual comparison?

• The before-and-after calculation of 1,000job losses

• A comparison with trends in other similarpublic sector ports (which might suggest anannual decline of 10 percent in cargo han-dlers as mechanization is introduced)


183

Evaluation is mainlyconcerned withimpact.

MO

DU

LE 7

Box 7.2: The Importance of ControlGroups—A Hypothetical Example

In the town of Abca, 1,000 workers were laidoff as a result of the closure of the ABC GasCompany. Based on random selection, 500

workers were given a severance package andthe other 500 were put through an intensiveretraining program in computer skills. All 1,000people were monitored over time. Three monthsafter the completion of the training, 400 traineeswere employed. This employment rate of 80percent for the treatment group was touted bymany as the impact of the training program.

However, Abcan evaluators cautioned againstusing only this figure to judge the success of theprogram. They wanted to compare this employ-ment percentage to that of the control group—those who did not go through training. It wasfound that 375 of the control group of 500 werealso employed three months after the treatmentgroup completed its training—an employmentrate of 75 percent. Hence, Abcan evaluatorsjudged that the true impact of the training pro-gram was 5 percent, not 80 percent.

Although this example makes many generaliza-tions—there was no selection bias or random-ization bias, those who got a severance pack-age did not enroll in any training or other relatedlabor programs, and so forth—it serves to illus-trate the importance of using control groupswhen evaluating the impact of labor programs.

Source: Adapted from World Bank, no date.

• A comparison with normal annual rates ofjob loss and job creation in private sectorports (a private sector counterfactual test)

• A comparison with cargo-handling laborbenchmarks in the most efficient ports inter-nationally against which the port can rea-sonably be compared.

In each case the counterfactual alternative providesthe “what-would-have-happened-if” comparison(in this case, what would have happened if PPI hadnot happened). Each comparison is, however, likelyto give rather different answers, and small differ-ences in the assumptions and comparisons beingmade can lead to very different conclusions. This iswhy the use of a counterfactual test with a veryclear definition of the assumptions being used is soimportant.

Assessing the Impact ofRedeploymentIn developing countries the evaluation of redeploy-ment programs, social safety net programs, andactive labor market programs has generally beeninadequate.

One reason for the neglect of evaluation might bethat properly assessing the impact of redeploymentpresents a significant technical challenge to evalua-tors everywhere. Undertaking a robust counterfac-tual analysis is particularly difficult because partici-pants in, for example, a training program may beselected or may self-select. Such selection biases candistort policy conclusions, and redeployment pro-grams are especially prone to these biases.Evaluators use two methodological approaches totackle these selection problems:

1. An experimental approach randomly assignsindividuals to enter a program. Thisapproach avoids many (though not all) ofthe selection problems of statistical method-ologies. It is difficult to implement, however,for practical reasons (that is, cost and thefact that only current or future programscan be evaluated by this approach) and forethical and political reasons (for example,

some workers are refused entry to the pro-gram). Few developing countries are likelyto implement such an approach. A recentillustration of such an evaluation is that of ajob-search assistance program for unem-ployed workers in the United Kingdom,where the evaluation identified a 6 percentlower unemployment rate among partici-pants five years after the initial program(Dolton and O’Neill 2002).

2. A statistical approach allows selection of theparticipant and nonparticipant groups afterthe redeployment program has started. Todeal with selection biases, complex econo-metric techniques are needed to reduce thebiases (elimination is not possible).Regression techniques and matched-paircomparisons are the principal statisticaltools. The main advantage of statisticalapproaches is that the evaluation can bedone at any time, provided that adequatelongitudinal data are available.

Dar and Gill (1998) summarized these alternativemethodological approaches in the context ofretraining programs. Theoretically, experimentaltechniques are more robust. However, the statisti-cal approach is more practical, although it can besubject to large biases that risk offering false con-clusions to policymakers. Matched-pair statisticaltechniques are preferable to regression-based tech-niques because (a) they offer the greatest potentialfor reducing differences between the participantand nonparticipant groups (other than the rede-ployment program), and (b) the results are easierfor nonstatisticians and policymakers to interpret.

The example of the evaluation of Mexico’s retrain-ing program for unemployed and displaced work-ers (box 7.3) illustrates some of the challenges ofconducting evaluations. Problems for evaluatorsinclude:

• Creaming: If program managers are evaluat-ed on the percentages of trainees who findemployment, then creaming may occur. Inthis situation managers actively select thebest or most-qualified trainees to inflate theprogram’s apparent success rate:


184

Redeploymentprograms indeveloping countrieshave rarely beensubject to rigorousevaluation.

Rigorous evaluationpresents a technicalchallenge toevaluators andrequires specialistskills.

MO

DU

LE 7

The analogy is to whole milk where the rich-est part, the cream, floats to the top and canbe skimmed off. Creaming is an issue inlabor market programs because if only themost able people get reemployment assis-tance, then the benefit to society of the pro-grams is not as great as it might be other-wise. Highly qualified program entrants havea good chance of becoming reemployed evenwithout the services offered in the program,while for less qualified applicants the pro-gram services might be the only realistic pathto employment (O’Leary 1999, p. 3).

To tackle this problem in evaluations, theright counterfactual test is needed. If theemployment rate of participants is com-pared with that of all displaced workers,then the apparent success of the programwill be inflated. Control or comparisongroups should therefore compare traineeswith other displaced workers who had simi-lar levels of qualifications and other observ-able attributes.

• Creating matched control or comparisongroups: If evaluations are undertaken sometime after the program is completed, itbecomes increasingly difficult to ensurematching between the treatment and the con-trol or comparison groups. In Mexico’sPROBECAT evaluation, for example, thecomparison group was taken from a separatedata set of an urban household survey ofworkers who were also unemployed at thetime that unemployed trainees entered thePROBECAT program. Such different datasets may not be fully comparable. Ideally,control or comparison groups should be indi-vidually matched, but adequate data for suchmatching are available only sometimes.

• Self-selection: In training programs whereindividuals choose (self-select) whether toenter the program, the problem of con-structing the counterfactual comparisonbecomes more difficult because those whoattend the program will be different fromthose who do not. If trainees volunteer forthe program because it offers a stipend, for

example, this can lead to selection biaseswhen evaluating the program.

• Dropouts: This is a related problem. Iftrainees drop out of the program when theyfind jobs, is that counted as a program suc-cess? Or does it simply show that traineesmerely participated for the stipend?

• Dead-weight loss: In later phases of PROBE-CAT, in-service training was provided bylocal employers. Government provided theworkers’ stipend, and the employers wererequired to hire at least 70 percent of thetrainees. “Dead-weight loss” refers to thefact that firms participating in the in-servicetraining would have hired some of the sameworkers anyway.

• Influence of the very existence of the pro-gram: The evaluation approach outlined inbox 7.3 used a conventional “differences-in-differences” approach, where the before-and-after earnings or employment changesfor participants in the redeployment pro-gram are compared with the before-and-after change for a similar group of nonpar-ticipants at a similar time. The approachassumes that the existence of the programitself is an external variable. Training pro-grams may function as a form of job searchfor many of their participants (Heckmanand Smith 1998). The decision to participatetherefore also needs to be controlled for inthe design of the evaluation.

• Displacement effects: If a program partici-pant improves his or her reemploymentchances at the expense of nonparticipants,then one person’s job may merely be takenby another. If this happens the program’soverall benefit to the economy as a wholemay be less than intended.

• Changes in program design: Programs oftenchange their design and approach duringimplementation. This is a problem for evalu-ators because (a) it compounds selectionproblems and (b) what they are evaluatingmay be seen as the “old” approach andtherefore not relevant.


185

Any evaluationneeds to correct forpotential selectionbiases.

MO

DU

LE 7

Given the importance often attached by govern-ments to redeployment, there is a good case forbetter evaluation. In a review of active labor mar-ket programs, Fay (1996) concluded that evalua-tion will be improved if:

• Evaluation is made compulsory in the pro-gram design phase. Most donor-funded pro-grams are subject—or potentially subject—to postevaluation. Although the benefits ofevaluations may not accrue to the govern-ment, they will improve the quality of thedatabase for other countries.

• Evaluations are more rigorous. Evaluationof the overall effects of labor programs iscomplex. The design of the evaluationmethodology requires specialist economicand evaluation skills.

• Evaluations are undertaken by nongovern-ment agencies. This has two benefits: gov-ernments do not need to use scarce profes-sional resources; and if the results comefrom an independent organization, they willprobably carry more weight.

• The period of evaluation is extended. Impactson workers of, for example, retraining maynot be observable shortly after the end oftraining. It may be valuable to wait longerafter the program before beginning the evalu-ation.

Evaluation studies of active labor programshave been conducted in middle-incomecountries at costs of about US$150,000(Fretwell 2002), which is a relatively modestamount compared with the levels of expen-diture incurred.


186

MO

DU

LE 7

Box 7.3: Example of a Redeployment Evaluation—PROBECAT, Mexico

In 1984, as a response to a growing economiccrisis, the government of Mexico established alabor retraining program for unemployed and

displaced workers—Programa de Becas deCapacitación para Trabajadores, or PROBECAT.Revenga, Riboud, and Tan (1994) reported animpact evaluation analysis. The evaluation setitself four clear questions. First, what is theimpact of training on the subsequent employ-ment experiences of trainees? Second, doestraining increase the speed with which traineesmove from unemployment to employment?Third, conditional upon finding employment, whateffect does training have on the monthly earn-ings, work hours per week, and hourly wages oftrainees? Fourth, do the monetary benefits fromprogram participation outweigh the costs of pro-viding retraining for the unemployed?

PROBECAT was a large program. At the time ofthe evaluation it had trained 251,181 unemployedpeople and provided 9,268 courses since 1987.During the training period (usually three months),program participants received a stipend equal tothe minimum wage. Vocational courses wereorganized to respond to the needs of the locallabor market and were designed to redress localshortages of workers with particular skills. Theseneeds were determined through periodic studiesof local labor market conditions.

Not everyone was eligible to participate inPROBECAT. The selection procedure gave vari-able weights to different criteria, including thenumber of economic dependents, attainment ofcertain levels of basic education, prior work expe-rience, and unemployment of less than threemonths. The weighting scheme was quite com-plex and nonlinear, and only individuals with atotal composite score exceeding a threshold levelwere eligible to join the program. In addition, par-ticipants had (in theory) to be between the agesof 20 and 55 and be registered as job seekers atthe local state employment office. This nonran-dom selection of individuals into PROBECATposed potentially serious measurement problemsfor the evaluation of the training program.

The evaluation approach taken was to adopt astatistical methodology to account for the selec-tion bias in the program, and to compare thepost-training labor market experiences ofPROBECAT trainees with those of a comparisongroup—a matched sample of unemployed peo-ple who were eligible for but did not participate inthe training program. The evaluation found thatparticipation in the training program decreasedthe period of unemployment for men and womentrainees and increased the monthly earning ofmen but not of women.

Source: Revenga, Riboud, and Tan 1994.

In addition, the costs and benefits of redeploymentneed particular attention because they are oftenneglected in evaluations. Although evidence ispatchy (Dar and Gill 1998), cost-benefit assess-ments indicate that:

• Large-scale retraining programs may not beas effective as other measures, such as job-search assistance.

• Targeting programs may improve their rele-vance and effectiveness. In some cases (forexample, in Hungary) redeployment trainingis better focused on relatively disadvantagedjob seekers, whereas other evaluations (suchas PROBECAT in Mexico) suggest that theprogram is more cost-effective if focused onbetter-educated and more experienced jobseekers.

Box 7.4 offers some key indicators that could beused both for interim (gross impact) monitoring ofredeployment programs (where usually there willbe batches of trainees) and for subsequent netimpact evaluations.

Assessing the Effects on Workers’WelfareOne of the simplest approaches to estimatingworker welfare loss was that of Galal and others(1994) in their evaluation of the impact on work-ers of one form of private participation (privatiza-tion). As Birdsall and Nellis (2002, p. 31) pointedout, that approach was “simple, completely openin noting the short cuts taken and derives a usable,quantified answer to a most complex question”—in their case, whether workers had been worse orbetter off following severance. To illustrate theirapproach they simply assumed an average wage inthe economy (for example, $250 per month), andif it took workers 10 months to find a job, thenworkers receiving a severance package of $2,500would be no worse or better off.

Birdsall and Nellis 2002.

This simple approach does not provide a robustcounterfactual test. In practice, however, it can bedifficult to eliminate other factors influencing theimpact on workers. As Rama and MacIsaac (1999,pp. 101–2) noted:

The most straightforward indicator of the lossexperienced by displaced employees is thechange in their annual earnings, excludingreturns from invested compensation. This indi-cator could be criticized on the grounds thatearnings before separation do not provide anappropriate counterfactual. A case could bemade that the appropriate comparison is withthe earnings these employees would havereceived had there been no downsizing. If thesituation prior to downsizing was unsustain-able, it could be argued, earnings would havedeclined in any event. Alternatively, if the situ-ation was sustainable, some of these employ-ees would have gotten pay raises and promo-tions in the 15 months elapsed since separa-tion. More generally, the appropriate compari-son would be between the lifetime earningsprofile after separation from [the enterprise]and the corresponding earnings profile in thecase of no separation. But this comparisonwould require heroic assumptions, so that it issafer to stick to observed earnings before andafter separation.


187

Evaluations shouldinclude assessmentof the costs andbenefits ofredeployment. M

OD

ULE

7

Box 7.4: Possible Cost and BenefitIndicators for Redeployment Programs• Average cost per entrant into counseling or

training—disaggregated among differenttypes of counseling or training

• Average cost per trainee employed

• Percentage of trainees employed or self-employed

• Percentage of trainees engaged in the voca-tion of training

• Average monthly wages/net incomes oftrainees (absolute and relative to preprogramincomes)

• Average household incomes (to assesseffects on other family members and thehousehold as a whole)

More robust evaluations of effects on worker wel-fare, however, can use the same experimental andstatistical methodologies described above for evalu-ating redeployment impacts. Unfortunately, therehave been few longitudinal tracer studies on whatactually happens to displaced employees over time.

A further difficulty in assessing impacts on work-ers’ welfare is the wide range of workers’ outcomesfollowing displacement:

• They retire and cease looking for paidemployment or income-earning opportuni-ties.

• They cannot find work or incomes andremain in long-term unemployment.

• They find alternative permanent employ-ment.

• They find alternative short-term, contractu-al, or informal employment.

• They chose to become self-employed as indi-viduals or start a microenterprise.

• They launch a formal small business withpotential for growth.

• They expand existing income-earning activi-ties that they were already running whileemployed in the enterprise (either “moon-lighting” or “daylighting”).

• They migrate out of the region to find jobsor to return to rural communities where thefamily home is based.

If workers migrate to find new employment, fol-low-up evaluation is more difficult and more cost-ly, and it is likely that such workers will not becaptured in subsequent evaluations. In a survey of5,334 workers from Brazil’s railway, 1,217 work-ers could not be found because they moved with-out a trace (Estache, Schmitt de Azevedo, andSydenstricker 2000).

In a follow-up survey of 675 former workers inBrazil that was conducted two to three years afterretrenchment, it was found that although 53 per-cent were earning less than when they were at thestate enterprise, 23 percent were making a better

living. In general the dispersion of wages wasgreater in this survey than one conducted ninemonths earlier. (There is no information on thesampling methods in these surveys, however.)

Most cost-benefit assessments make fairly simpleassumptions regarding the consequences of dis-placement—a number of months of unemploymentat a wage of zero, followed by a wage of, forexample, 60 percent of the previous wage. Withoutmore detailed tracer studies the impacts of morecomplex outcomes may not be well known.Nonetheless, it is clear that many displaced work-ers move into self-employment. Table 7.5 showsone example from Turkey, where nearly one-fifthof workers displaced during privatization usedtheir severance money to enter into self-employ-ment. A January 1998 survey of displaced workersin Brazil’s federal railway found that “over halfwork on their own and 20 percent have openedtheir own business. Only 18 percent are employeesand four percent are civil servants” (Estache,Schmitt de Azevedo, and Sydenstricker 2000, p.18). A tracer study in Ghana found that nearly 70percent of displaced civil servants went into self-employment (table 7.6). In addition, there may beimpacts on others. Although SOE workers are rela-tively well paid, those benefits may be shared withhouseholds and extended families.

Assessing Overall PPI BenefitsEvaluating the success of a labor program requiresthat initial objectives be revisited. It asks, whatwere the initial objectives? and did the programmeet those objectives?

An evaluation can be made at two levels:

1. Evaluation against the specific objectives ofthe labor program itself: This is the focushere, especially the impact of redeploymentprograms on workers’ incomes and the peri-od of unemployment, and the impact oflabor adjustment on workers’ welfare.

2. Evaluation of the contribution of the laborprogram to achieving the wider policy objec-tives of PPI: The effects of a labor programmay go well beyond the consequences for indi-


188

MO

DU

LE 7

vidual displaced workers. Table 7.7 indicatesthe scope of a comprehensive assessment—which would include impacts on government,consumers, investors, and labor unions, as wellas workers themselves. Box 7.5 summarizesinformation on the extent to which labor vari-ables influence privatization prices.

MONITORING LABOR PROGRAMSMonitoring of the labor program is a managementtask, but one that is often overlooked and neglected.

Monitoring differs from evaluation in that it is prin-cipally a management function, which typically


189

More detailed tracerstudies are needed.

Many workers moveinto self-employment.

MO

DU

LE 7

Table 7.5: Turkey—How Workers Used Severance Compensation(percentage of workers)

Petrochemical workers Cement workers Total Use of severance money (n = 682) (n = 563) (n = 1,245)

Established own business 12.8 22.0 17.0

Used for daily expenses 31.4 28.2 30.0

Lent money 6.0 1.7 4.0

Placed time deposit in a bank 22.1 5.4 11.2

Bought a house 40.0 36.1 38.2

Bought gold or foreign exchange 9.4 18.3 13.3

Bought treasury bills 1.6 0.4 1.1

Bought securities 0.9 0.7 0.9

Used interest income for daily expenses 7.1 5.0 6.2

Used rental income for daily expenses 4.2 3.4 3.8

Bought a car 18.1 10.0 16.7

Bought land 3.5 1.4 2.6

Note: Based on an interview where respondents were faced with a number of possible choices and asked to choose as many as applica-ble. So, for example, around 40 percent of petrochemical workers stated that they bought a house with their severance compensation.

Source: Tansel 1996.

Table 7.6: Preferred Employment Status of Redeployed Civil Servants—Ghana

Preferred Agriculture Nonagriculture Totalemployment status Number Percent Number Percent Number Percent

Self-employment 1,124 72 782 65 1,906 69

Cooperative 420 27 260 21 680 24

Private wage employment 20 1 164 14 184 7

Government 2 0 6 0 8 0

Subtotal 1,566 100 1,212 100 2,778 100

No preference (number) 95 n.a. 837 n.a. 932 n.a.

Total sample size (number) 1,661 n.a. 2,049 n.a. 3,710 n.a.

n.a. Not applicable.

Note: Based on sample of workers opting for retraining.

Source: Alderman, Canagarajah, and Younger 1994.


190

MO

DU

LE 7

Table 7.7: Assessing Labor Programs—A Checklist of Potential Effects on Different Stakeholders

For Type of For For For consumers For impact government unions employees and customers investors

Positive •Reduced •Greater job •Salary improve- •Faster access to • Improved financial effects subsidies or security (but ments for improved PPI performance

net costs of for fewer retained services •Reduced costsproviding employees) workers •Evidence of •More flexible labor PPI services •Stronger role, •Changes in growth in contracts

•Time advan- if consulted labor contracts supply of • Improved labor tage from and partici- that affect services productivityfaster com- pate in (improve or (e.g., access pletion of PPI preparation reduce) non- to water) or transaction, of the labor wage benefits demand for and faster program services implemen- (number of tation of passengers investment on trains)or service •Reduced costs improvements to business

•Revenues (e.g., telecom-from PPI munications, transaction transportation)(concession •Reduced tariffs or privatiza- to consumers tion receipts) (services)

•Increases in •Service quality tax revenue improvementsfrom private operators

•No disruption of service (power sup-plies, port operations)

Negative •Political •Loss of •Loss of •Increased •Higher wage or (adverse) costs (if membership salary and tariffs (if benefits costs if effects disputes) numbers other tan- there are negotiated upward

•New incre- •Reduced gible and labor cost by labor prior to PPImental costs bargaining intangible increases) •Less flexible labor

•Financial power benefits for contractsloss if displaced •Loss of valuablerehiring workers skills if adversetakes selectionplace


191

MO

DU

LE 7

Box 7.5: Impact of Downsizing on PPI Prices

The very high levels of downsizing in infra-structure enterprises described in module2 suggest that infrastructure enterprises

are qualitatively different from other privatizationsor PPI schemes. Unfortunately, there is little evi-dence of the impact of downsizing on PPI pricesreceived from investors. In part this is becauseinvestor behavior is inherently complex, and inpart because downsizing is often part of a widerpackage and investor responses to downsizingmay be confounded by other changes (such asrevised labor contracts or relations). In somecases, without work force restructuring it isunlikely that any investor will be found. The dra-matic work force restructuring in Mexico’s air-lines, described in module 4 (Box 4.9) is oneexample. Another is Argentina’s railway, whichRamamurti (1997) characterized prior to privatiza-tion as a “lemon”—an enterprise not attractive toa private investor because it was in a stagnant ordeclining market with poor profit prospects—butnoted that:

The government did several things to turn FA[Ferrocarriles Argentines] from a lemon—notinto a plum—but into a much sweeter proposi-tion than before (a grapefruit?) Four govern-ment steps in that direction were: breaking theFA’s unions, picking up the tab for downsizingits work force, splitting up the company intosmaller parts and then leasing rather than sell-ing its assets. In addition, the unions agreed togreater flexibility in the deployment of workers,and to negotiate contracts with private ownersthat would increase rail’s competitiveness (p.1982).

One attempt to review alternative factors influ-encing privatization prices is that of López-de-Silanes (1997). Using a database of all 236Mexican privatizations between 1983 and 1992,he assessed the factors that influenced privatiza-tion price (as measured by a privatization quo-tient [PQ]). He found that “labor issues play acentral role in explaining privatization prices” (p.997), and that after accounting for endogeneity,reduction in the labor force increased privatiza-tion prices: “the net effect of a 20 percent reduc-tion in the labor force before privatization is a 24percent increase in PQ, evaluated at the predict-ed mean” (p. 1015). Moreover, union relation-ships were important: union contract renegotia-

tion improved privatization prices (although thiswas not statistically different), and industrial dis-putes strongly depressed privatization prices:“one of the strongest results...is that an addition-al strike in an SOE leads to a 19 percent reduc-tion in the net price evaluated at the mean pre-dicted PQ” (p. 997).

A more recent assessment by Chong and López-de-Silanes (2002) undertook follow-up surveys of308 privatized enterprises taken from a globaldatabase of privatizations. Using dummy vari-ables for various labor downsizing policies, theyfound that labor downsizing did little for net pri-vatization prices. The analysis, however, wasunable to differentiate between large levels ofdownsizing—as occur in most infrastructure pri-vatizations—and more modest levels of downsiz-ing (only a quarter of the survey respondents pro-vided any numerical information).

Evidence of the complexity of investor behaviorcomes from private sector work force restructur-ing. The common assumption is that stock mar-ket prices will rise following downsizing. Abrahamand Kim (1999) reviewed a number of studies onthe effects of downsizing on investor behaviorand found that the evidence is inconclusive. Theirown study of 381 firms found that both layoffannouncements and employment guaranteeannouncements lead to reductions in stock mar-ket share prices. They suggested that investorresponse depended on the net result of four pos-sible effects on investor behavior following down-sizing:

1. A positive cost-saving effect (downsizingreduces cost of production)

2. A positive efficiency effect (downsizingimproves overall firm efficiency)

3. A negative industrial relations effect (downsiz-ing leads to poorer labor relations)

4. An ambiguous signaling effect (for a firm ingood shape, layoffs indicate a positiveresponse to changing circumstances; for firmsin poor shape, layoffs confirm that firm per-formance is poor or even worse than expect-ed).

Note: PQ is government’s net privatization price afterrestructuring, adjusted by the percentage of companyshares sold plus total liabilities at the time of privatization,divided by the total assets of the company at the time of privatization.


192

MO

DU

LE 7

Box 7.6: Monitoring and Evaluation DefinedMonitoring is the continuous assessment ofprogram implementation in relationship toagreed schedules and the use of program out-puts by beneficiaries.

Evaluation is the periodic assessment of therelevance, performance, efficiency, and impactof the program in relationship to stated goals ofthe program.

involves both review of performance against targetperformance indicators and foreward-looking fore-casts (see box 7.6). Effective monitoring of thelabor program is integral to good management bythe implementing agency. It provides the agencycontinuous feedback on implementation and identi-fies both successes and problems as early as possibleto facilitate timely adjustments to project operation.

The potential benefits of monitoring assessmentsare large: one study in Tanzania found that theinformation from monitoring studies could havesaved the government up to US$7 million duringthe course of retrenchment of about 5,000 stateenterprise workers.

Given the potential scale of infrastructure workforce adjustment programs (thousands of workers,perhaps tens of millions of dollars), making thecase for monitoring should seem unnecessary. It isclear, however, that when the work force restruc-turing plan is in place, some governments fail tomonitor it adequately.

Why is this so? Five possible reasons:

1. Ownership: a general problem of all moni-toring (and evaluation). If there is no cus-tomer for monitoring information, thenmonitoring systems are unlikely to work.

2. An (incorrect) belief that implementation is theeasy part: When the big decisions have beenmade on the severance package, the mecha-nism of restructuring, and the scope of restruc-turing, the implementing agency and govern-ment officials may feel that the main chal-lenges have been resolved. Unfortunately, allthe evidence suggests that effective work force

restructuring needs strong, active ownershipand strong monitoring during implementation.

3. Compartmentalization within government:Institutional gaps between policy and imple-mentation responsibilities for work forcerestructuring can result in a lack of policycoherence and coordination

4. The desire for a clean break: There may bea concern that follow-up surveys can lead toraised expectations by workers of what gov-ernment may be able—or willing—to do forthem. Particularly if politicians have agreedgenerous severance terms, there may be adesire to minimize all contact with workersfollowing severance.

5. Exaggerated expectations: There is a tenden-cy among politicians and governmentspokesmen to convey exaggerated expecta-tions of what redeployment programs canachieve. Hence, there is little subsequentinterest in monitoring or evaluation if doingso might reveal a different story.

Where day-to-day implementation of some elementsof work force restructuring is delegated to the enter-prise or to other agencies, monitoring systems areparticularly valuable. Here are some key steps thatthe implementing agency can take to help:

• Define and promulgate a set of monitoringindicators, against which the implementationmanagers report, and indeed on which theimplementing agency reports. Possible indica-tors are set out in box 7.7. These indicators ontheir own will not guarantee effective monitor-ing. But the selection of an indicator can helpto ensure that progress is made in that area.

• Require regular staffing reports from theenterprise. These will be especially relevantwhere there are high risks of adverse selec-tion or rehires or a history of labor hoard-ing, or where the PPI process is likely to beslow. A sample monitoring report providedon the accompanying CD-ROM illustrates aformat for monthly or quarterly reporting ofwork force change due to natural retirementand attrition, soft options (such as transfers,

administrative leave, and removal of ghostworkers), early retirement and voluntarydeparture, rehiring, and new hiring.

• Clearly define the objectives and institution-al responsibility for monitoring. This caninclude the creation of special units withinthe implementing agency to monitor compli-ance with labor (and other) conditions inPPI contracts. In Germany the privatizationagency—the Treuhandstadt—set up a spe-cial department responsible for enforcingprivatization contracts using penalties andlegal action; when the agency was dissolved,those monitoring tasks were transferred toanother government body.

Tools (on the CD-ROM)Spreadsheet for analysis of labor projects

Labor program monitoring report

Example terms of reference for impact evaluation

Monitoring and evaluation performance measures forjob-search assistance services

Additional Material (on the CD-ROM)

Birdsall, Nancy, and John Nellis. 2002. “Winners andLosers: Assessing the Distributional Impact ofPrivatization.” Working Paper 6. Center for GlobalDevelopment. World Bank, Washington, D.C.

Chen, Yi, and Ishac Diwan. 2000. “When theBureaucrats Move out of Business: A Cost-BenefitAssessment of Labor Retrenchment in China.”Policy Research Working Paper 235. World Bank,Washington, D.C. (Sets out a methodology to esti-mate the costs and benefits of labor retrenchmentin state-owned industrial enterprises in China.)

Dar, Amit, and Indermit S. Gill. 1998. “EvaluatingRetraining Programs in OECD Countries: LessonsLearned.” The World Bank Research Observer13(1):79–101. (Summarizes alternative evaluationtechniques and also reviews the findings of evalua-tions of retraining in OECD countries.)


193

Governmentscommonly fail tomonitor laborprograms effectively.

Insufficientinvestment inmonitoring cansuggest that the sizeof theimplementationchallenge has beenunderestimated.

.

MO

DU

LE 7

Box 7.7: Severance and Redeployment: Some Monitoring Indicators

The key objectives of monitoring are toassess overall progress and performanceof the work force restructuring program, to

identify the extent of adverse selection, and toreinforce accountability in the use of publicfunds. Examples of monitoring indicators are:

Severance Indicators

• Numbers of workers displaced by differentmechanisms (early retirement, voluntary depar-ture, mandatory departure, administrativeleave)

• Disaggregation of displaced workers by age,grade, years of service, gender, ethnic group,region

• Disaggregation of displaced workers byskill/job description/operating unit

• Staffing ratios (based on relevant benchmarks,such as those described in module 3)

• Total wages and staff-related costs (from finan-cial management reports)

• Total employment changes in the enterprise—inflows and outflows (monthly or quarterlyreports).

• Time between workers’ termination and receiptof monies

• Time between termination and receipt of audit-ed accounts for disbursement

• Total and average severance payments

• Percentage of disbursements with queries/out-standing (sorted by reason)

• Numbers of appeals filed by workers againstseverance period.

Redeployment Indicators

• Numbers and percentages of displaced work-ers who receive counseling of different types,job-search assistance, training, or retraining

• Disaggregation of displaced worker by age,grade, years of service, gender, ethnic group

• Number (percentage) of dropouts from differ-ent training programs/percentage completingtraining

• Duration of training programs (training days perworker) disaggregated by type of training andby worker characteristics.

Not all of the above indicators are needed. Ingeneral, use as few indicators as necessary.

Haltiwanger, John, and Manisha Singh. 1999.“Cross-Country Evidence on Public SectorRetrenchment.” The World Bank EconomicReview 13(1):67–88. (Reviews evaluations of 41downsizing programs, some in state enterprises andsome in the public sector as a whole.)

Rama, Martin. 1997. “Efficient Public SectorDownsizing.” Policy Research Working Paper1840. World Bank, Washington, D.C. (Provides anoverview of public sector downsizing, and com-ments on approaches to financial and economiccost-benefit analysis of downsizing.)

Rama, Martin, and Donna MacIsaac. 1999.“Earnings and Welfare after Downsizing: CentralBank Employees in Ecuador.” The World BankEconomic Review 13(1):89–116. (Describes anevaluation of the impact of downsizing in a state-owned bank.)

Revenga, Ana, Michelle Riboud, and Hong Tan.1994. “The Impact of Mexico’s RetrainingProgram on Employment and Wages.” The WorldBank Economic Review 8(2):247–77. (Describes animpact evaluation of a retraining program forunemployed and displaced workers.)

Ruppert, E. 1999. “The Algerian RetrenchmentSystem: A Financial and Economic Evaluation.”The World Bank Economic Review 13(1):155–83.(Summarizes an analysis of the retrenchment pro-gram in Algeria that combined both severance andunemployment benefits.)

World Bank. 1996. “Designing Project Monitoring andEvaluation.” Operations Evaluation Department,Lessons and Practices 8. Washington, D.C. (Availableto download from www.worldbank.org.)

Web sitesInternational Institute for Labour Studies.

www.ilo.org/public/english/bureau/inst/index.htm.(Conducts and publishes research on labor marketsand labor policies.)

World Bank Operations Evaluation Department:www.worldbank.org/oed.

Other Material and SourcesBaker, J. L. 1999. Evaluating the PovertyImpact of Projects: A Handbook for

Practitioners. Washington, D.C.: World Bank.

(Outlines tools to evaluate project impacts, withextensive case studies of a wide range of evalua-tions.)

Galal, Ahmed, Leroy Jones, Pankaj Tandon, and IngoVogelsang. 1994. Welfare Consequences of SellingPublic Enterprises. New York: Oxford UniversityPress. (Main volume is out of print; summary vol-ume is still available from World Bank Publications,www.worldbank.org.)

Grubb, W., and P. Ryan. 2000. The Roles ofEvaluation for Vocational Education and Training.Geneva: International Labour Office. (Focuses onvocational training but provides an overview of eval-uation techniques and methodologies.)

López-de-Silanes, Florencio. 1997. “Determinants ofPrivatization Prices.” Quarterly Journal ofEconomics 62(4):965–1025. (Reviews the determi-nants of privatization prices using a data set of allMexican privatizations in the period 1983–92.)

O’Leary, Christopher J. 1995. “PerformanceIndicators: A Management Tool for Active LaborProgrammes in Hungary and Poland.”International Labour Review 134(6):728–51.(Outlines potential performance indicators.)

O’Leary, C., A. Nesporova, and A. Samorodov. 2001.Manual on Evaluation of Labor Market Policies inTransition Economies. Geneva: InternationalLabour Office. (Discusses various labor market pro-grams in transition countries, evaluation methodolo-gy, and how to use the evaluation results.)

Schmid, G., J. O’Reilly, and K. Schomann. 1996.International Handbook of Labor Market Policyand Evaluation. Cheltenham, United Kingdom:Edward Elgar Books. (Outlines the various method-ological approaches adopted in evaluation research,presents cross-country evaluation findings, andpresents insight into institutional frameworks andsystems of monitoring and evaluation.)

Tansel, Aysit. 1996. “Workers Displaced Due toPrivatization in Turkey: Before versus afterDisplacement.” Paper presented at World BankConference on Public Sector Retrenchment andEfficient Compensation Schemes, November 6–7.Washington, D.C. (Illustrates a methodology forevaluating the impact on workers of a work forcerestructuring program.)


194

MO

DU

LE 7

module monitoring and evaluation of labor programs

Documents