adding it up - archive · a crucial “challenge role” to play in ensuring that key policies and...
TRANSCRIPT
A PERFORMANCE AND INNOVATION UNIT REPORT – JANUARY 2000
CABINETOFFICE
Adding it upImproving Analysis & Modelling in Central Government
1
CONTENTS
Foreword 3
1. Executive Summary 5
2. Introduction 8
3. What are the problems? 12
4. Analysis in an ideal world 22
5. Stimulating the demand for good analysis 25
6. Planning and improving the supply of good analysis 34
7. Integrating analysis with policy making 39
8. Recruitment and retention of good analysts 45
9. How to get more and better data 53
10. Improving the quality of microeconomic 61
modelling in Government
11. Making better use of the external world 70
12. Implementation strategy 74
Annex A1: The PIU 84
Annex A2: The Steering Group and the Project Team 86
Annex A3: Main findings from the case studies 87
Annex A4: Lessons from overseas experience 99
Annex A5: Interview record and Glossary 103
List of sources 106
Contents
BY THE PRIME MINISTERThe job of government is to meet the needs and aspirationsof the people. That depends on deciding on the right policiesand then delivering them effectively.
Getting policies right depends on accurate data and analysis.There are about 1,800 specialists in Whitehall, includingeconomists, statisticians, researchers and actuaries, who providethat analysis and modelling. They do an excellent job. But aseverywhere else in government, there is always room forimprovement in how they work and the way the rest ofgovernment makes use of the information they provide.
This means learning from mistakes, seeing what works best,and making comparisons to the highest benchmarks in therest of the UK and internationally.
The Modernising Government White Paper we published lastyear set out this Government’s commitment to policy-makingbased on hard evidence. And as in education, or NHS reforms,or fighting crime, we must always be looking at the outcomesof policies – the benefits in people’s lives – not the process.
That is why I asked the PIU to examine how we could achievethis, as part of the wider reforms of the Civil Service now underway. Their report not only shows the size of the potentialbenefits. It also sets out new ways to ensure that analysis andmodelling is given due weight in policy advice to ministers andsenior managers, and that analysis, like policy itself, is properlyjoined up.
The conclusions in ‘Adding it up’ constitute a major programmeof reform. Putting this into practice successfully will mean betterdecision-making and better government in the years ahead.
Tony Blair
FOREWORD
Foreword
3
Executive Summary
5
Key messages1.1 This report forms an integral part ofa series of reviews and initiatives to preparethe public service for the 21st century. Itsconclusions set out a comprehensive andcoherent programme for creating theconditions in which rigorous analysis isroutinely demanded and delivered.Implementation of this programme willbe a key step towards the commitmentto evidence-based policy made in theModernising Government White Paper (1)and developed in the Cabinet Office paperon Professional Policy Making (2).
1.2 A great deal of good analytical work iscarried out in central Government. But thereis also considerable scope for improvement,to bring analysis in the UK up to the highestinternational standards. There needs to be afundamental change in culture to place goodanalysis at the heart of policy-making. Thiswill require:
• leadership from Ministers and seniorofficials;
• openness from analysts and policy makers;
• better planning to match policy needs andanalytical provision;
• spreading best practice across departmentsand professions;
• innovative solutions to recruit and retainthe best people.
Leadership1.3 Better analysis requires commitmentthroughout Government. Ministers andsenior civil servants have a crucial leadershiprole in expecting and demanding soundlybased analysis in support of policy. They needbetter understanding of the importance ofanalysis and how it can contribute to gooddecisions and policy outcomes.
1.4 Policy makers in central Governmentalso need a better understanding of whatanalytical colleagues can offer. Such anapproach is operated successfully in theTreasury where all policy makers are requiredto have reached minimum standards ofproficiency in analytical skills if they wishto pursue opportunities for promotion.
1.5 This report concludes that:
• training for new Ministers and for seniorcivil servants run by Centre forManagement and Policy Studies (CMPS)should emphasise the importance ofanalysis for evidence-based policy;
• individual departments should define theneed for training for policy makers andpursue this intensively to redress any biasagainst quantification and analysis.
Openness1.6 Openness is a very powerful incentivefor encouraging the highest standards ofanalysis in support of policy making becausethe results then have to be publiclydefensible.
1. EXECUTIVE SUMMARY
1.7 This report concludes that:
• each department should produce an(annually reviewed) analytical strategyas part of its normal business planningprocess to encourage input, challenge andfeedback from experts and stakeholdersoutside Government;
• long-term quantitative models should bepublished and subject to external audit.
Planning1.8 Analysis needs to be jointly plannedby policy makers and specialists to ensurethe Government’s key policies, now and inthe future, are underpinned by the rightanalytical effort. The Treasury and theCabinet Office are well placed to spot gapsor deficiencies in departments’ analyticalstrategies. These central departments havea crucial “challenge role” to play inensuring that key policies and programmesare underpinned by good analysis.
1.9 The report concludes that:
• as part of the Spending Review process,the Chief Economist in the TreasurySpending Directorate should reviewdepartmental analytical strategies toidentify deficiencies, gaps and overlaps;
• to perform their challenge role effectively,central departments should undertake areview of their analytical capability.
Spreading best practice1.10 Government should know what itknows. Better management and organisationof knowledge within Government – one ofthe roles to be taken forward by the CMPS –would help to communicate analytical resultsand techniques across departments andspecialisms.
1.11 The report highlights a number of areaswhere disseminating best practice wouldimprove the quality of analysis:
• more and better use of pilots to test theimpacts of policies before national roll-out;
• making better use of the large amountsof data Government routinely collects;
• communicating clearly across Governmentwhat data are available;
• better networking between specialistsin Government.
Employing the best people1.12 To obtain the best analysis,Government needs to employ or to accessthe best analysts. The market place for goodanalysts is, however, highly competitive andincreasingly international.
1.13 The report concludes that Governmentshould take steps to make itself moreattractive to high quality analysts througha variety of routes:
• greater use of personal promotion to showthat high quality expertise is valued;
• more movement between generalist andspecialist posts within the Civil Service andgreater use of secondments;
• an urgent examination of the case forrewarding good analysis with higher pay.
Implementation1.14 The changes set out in this report willbe overseen by an Implementation Group,chaired by the Head of the GovernmentEconomic Service (GES). This group willmonitor the progress made by those withthe primary responsibility for implementingthe report’s conclusions. There are threekey players:
Addi
ng It
Up
6
• the CMPS will promulgate better useof analysis in policy formulation. Thedevelopment and implementation ofanalytical strategies in departments willbe a key element in its programme ofdepartmental peer reviews of policymaking;
• an expanded central microeconomicsteam in the Treasury, which will providethe secretariat for the ImplementationGroup;
• Permanent Secretaries, who as part ofthe Civil Service Reform process will takeforward the proposals about betterbusiness planning and bringing in andbringing on talented specialists.
Executive Summary
7
What is analysis and modellingand why does it matter?2.1 In the context of the activities ofGovernment, analysis involves theexamination and interpretation of data andother information, both qualitative andquantitative, to provide insights to improvethe formulation of policy and the deliveryof services. Modelling (a subset of analysis)is about establishing causal and formalmathematical relationships between variables.
2.2 Rigorous analysis and, whereappropriate, modelling is in the best interestsof both Ministers and senior officials. Theylead to better decisions and improved policyoutcomes. Without soundly based analysisand modelling, those involved in theformulation of policy and the delivery ofservices will work in the dark. As a result,the pace of reform may be slow.
What are we trying to do?2.3 The Modernising Government WhitePaper (1) and the Cabinet Office report onProfessional Policy Making (2) emphasisedan evidence-based approach to policymaking. The use of soundly based analysisand (where appropriate) modelling is a keystep towards this goal.
2.4 It was originally intended that this projectshould focus on the use of microeconomicmodelling in central Government. But – asbecame evident during the project –microeconomics is only one among severalanalytical disciplines that bear on policydevelopment. And modelling represents justone form of analysis. All forms can be equallycritical to evidence-based policy. Theseconsiderations led to a broadening of theoriginal remit for the project.
2. INTRODUCTION
Summary
There are many examples of good analytical work in Whitehall. Someare explored in detail in the case studies prepared for this report.
But there is also considerable scope for improvement – to embed thedemand for and supply of high quality analysis more firmly into theWhitehall culture.
This report aims to establish the framework for analysis that centralGovernment needs to take it into the next century.
Addi
ng It
Up
8
2.5 The present report covers the work ofmicroeconomists in Government in detail andin depth. It discusses the work of statisticians,actuaries, and social researchers, operationalresearchers and, on occasion, scientists wherethis overlaps with or touches on the work ofeconomists. It also covers issues that arecommon to specialists, such as recruitmentand retention of staff and joint working withpolicy-makers.
2.6 But the report does not discuss the useof techniques specific to social research (suchas the design of social surveys), OperationalResearch (such as cognitive mapping, softsystems analysis and much scenario modelling)or actuarial analysis. Macroeconomicmodelling and policy in the Treasury alsofall outside the scope of the project.
2.7 The report excludes the policies andactivities for which the devolvedadministrations in Scotland, Wales andNorthern Ireland are responsible. However,officials in the devolved administrations havebeen made aware of the PIU study and keptin touch with its emerging conclusions.
2.8 Against this backdrop, the objectives ofthe project have been:
• to map departments’ capabilities foranalysis and modelling and their accessto and use of existing data in key policyareas;
• to identify areas of strength and weakness(best and worst practice);
• to identify the reasons for those strengthsand weaknesses;
• to assess what links are needed betweenanalytical work in different policy areas toensure consistency of methods and dataand thus a coherent overall evidence basefor decision-making;
• to make cost-effective recommendationsfor change.
How is the report structured?2.9 This report is structured as follows:
• Chapter Three identifies the mainproblems surrounding analysis andmodelling in central Government, makinga distinction between problems arisingfrom the lack of demand for good analysisand the problems contributing toinadequate supply;
Introduction
9
Box 2.1. How Was the Project Carried Out?
In compiling this report the PIU team has drawn on:
• questionnaires sent to all departments to gather information on areas of policypriority in the immediate past and over the next few years; the analytical supportunderpinning these areas of policy; use of and access to data; long-term models;details of numbers of analytical staff and how they are organised;
• discussions with policy makers, economists and analysts in departments and at thecentre (Treasury, Cabinet Office, No. 10 and the Office for National Statistics (ONS));
• discussions with outside experts familiar with Government drawn from, for example,think tanks such as the Institute for Fiscal Studies (IFS), academia and the private sector;
• a series of detailed case studies from which lessons may be learned – details are at Box 2.2.;
• a review of best practice in the private sector, in the academic and “think tank”sectors, and overseas.
• Chapter Four sets out a ‘vision’ of the roleof analysis in an ideal world. It identifiesthe conditions which would need to besatisfied for there to be both a vigorousdemand for and a high quality supply ofanalysis;
• Chapter Five looks at the steps that mightbe taken to ensure Ministers and seniorpolicy makers demand good analysis tosupport major policy decisions;
• Chapter Six addresses how best to planthe supply of good analysis. The proposalsare aimed at ensuring that the rightpriorities for analysis are set betweendifferent policy areas, that economist andother professional resources are efficientlyand effectively deployed within andbetween departments and that cross-cutting analysis is carried out wherenecessary;
• Chapter Seven considers what practicalsteps need to be taken to promoteeffective joint working of various kinds –between professionals and policy makers,between different kinds of professionals;and between different departments;
• Chapter Eight considers the recruitmentand retention of good analysts, the roleof secondments and rewards andremuneration;
• Chapter Nine sets out a series ofproposals for improving the data availablefor modelling and analysis. This includesmeasures to encourage the improved useof data, to identify and fill data gaps andto define better the role of the centre(the Treasury, the Cabinet Office, No 10and ONS);
• Chapter Ten looks specifically at whatcan be done to improve the quality ofmodelling across Whitehall. A key themeof this chapter is openness – a willingnessto expose modelling to external scrutiny;
• Chapter Eleven explores howGovernment might make better use ofthe external knowledge pool throughstrengthening links between Governmentand academia, learning from the privatesector and contracting out analytical workwhilst retaining an intelligent customercapability in-house;
• Chapter Twelve sets out anImplementation Strategy;
• Annexes summarise the role of the PIU;give details of the project team and of theSteering Group which oversaw the project;summarise the findings from overseasvisits; give a full list of those interviewed;and summarise the case studies. Thedetailed case studies underpinning thisreport have been made available on thePIU web-site (www.cabinet-office.gov.uk/innovation).
Links with other initiatives2.10 This study is an integral part of a seriesof reviews and initiatives to prepare thepublic service for the 21st century. Inaddition to the Modernising GovernmentWhite Paper (1) and the Cabinet Office paperon Professional Policy Making (2), two keydevelopments are:
• the PIU report Wiring It Up (3), whichexamines how current accountabilityarrangements and incentive structures canbe reformed to facilitate more joined-uppolicy making and delivery. This studyhelps to put in place more joined-upanalysis in support of joined-up policy;
• the new Centre for Management andPolicy Studies in the Cabinet Office, whichwill act as a repository for all that is knownabout policy making and the delivery ofservices. It will play an important role inthe implementation of this report.
Addi
ng It
Up
10
2.11 The themes of all these reports –openness, joint working and the importanceof getting the role of the centre right – areechoed throughout this report.
Introduction
11
Box 2.2 List of Case Studies
Case Study 1 DETR National Road Traffic Forecast Model
Case Study 2 OFWAT Efficiency Models
Case Study 3 DETR Household Formation Model
Case Study 4 GAD Population Projection Model
Case Study 5 DSS Pensions Policy
Case Study 6 DSS Pensions Model (PENSIM)
Case Study 7 PSSRU Long Term Care Cost Model for DH
Case Study 8 DH Waiting Times Model
Case Study 9 LCD Civil Legal Aid Spending Model
Case Study 10 DfEE Labour Market Policy
Case Study 11 HMT Tax Benefit Model (IGOTM)
Case Study 12 DSS Policy Simulation Model (PSM)
Case Study 13 C & E Impact of Tobacco Smuggling
Case Study 14 DETR Model of Restructuring Social Rents
Case Study 15 DETR Air Quality Strategy
Case Study 16 FCO/DCMS Formulation of PSAs
Case Study 17 CO Electronic Delivery of Government Services
Case Study 18 HO Property Crime Models
3.1 This Chapter sets out the mainproblems that fieldwork for the project hasdiscovered with analysis and modelling incentral Government. It makes a distinctionbetween problems arising from the lack ofdemand for good analysis and those arisingfrom inadequate supply.
There is weak demand for good analysis3.2 The government is fully committed tothe principle that policies should be based on
evidence. This means policy should besupported by good analysis and, whereappropriate, modelling. This study has found,however, that demand for good analysis isnot fully integrated in the culture of centralGovernment. There are number ofconstraints on the use of analysis.
External pressures can constrain thescope for analysis3.3 The outcome of European Union (EU)and international negotiations can constrain
3. WHAT ARE THE PROBLEMS?
Summary
This Chapter identifies the main problems surrounding analysis andmodelling in central Government. These problems can be divided intotwo categories: those arising from the lack of demand for good analysisand those relating to inadequate supply.
On the demand side, the key issues are: the external constraints on thepolicy process stemming from e.g. international agreements or detailedmanifesto commitments; an occasional lack of interest in cross-cuttingquestions; more local disincentives to analysis e.g. where departmentshave a sponsorship role; and the tight deadlines associated with policydevelopment.
On the supply side, the key issues are: shortcomings in business planning;inefficiencies in joint working; and difficulties in recruiting and retainingthe best analysts.
These problems are reflected in the demand for and supply of both dataand modelling in central Government.
Addi
ng It
Up
12
domestic policy. For example, the UK nowhas a target to reduce carbon emissions by12.5% by 2008–2012 as a contribution toan EU target of an 8% reduction. Once suchagreements or targets are set, the role ofanalysis is largely to work up detailed policyoptions for implementation.
3.4 Detailed manifesto commitments canalso constrain a Government’s scope formanoeuvre. They are usually seen as bindingon the Government.
3.5 More often – and more helpfully –manifesto commitments do not take theform of specific pledges but are couchedin broad strategic terms (e.g. to develop astakeholder pension). The role of analystsand policy makers in Government is thento translate these broad goals into detailedand workable policy options.
The political timetable doesn’t matchthe timetable for analysis3.6 Ministers can set very tight deadlinesfor the delivery of analytical work. They canthemselves be under intense politicalpressures to develop quick solutions to policyproblems. At best, policy will generally bedeveloped across a 6-9 month timespan. Butresearch programmes and the developmentof microeconomic models can take muchlonger to complete.
A reluctance to collect and analyse data3.7 The direction of policy will often bedriven by a strong a priori conviction. Thisis beneficial when strong convictions areharnessed to rigorous and detailed analysis ofpolicy options. It can be less helpful if it leadsto a lack of interest in research and evidence.In departments where all research that iscommissioned is published, there is adisincentive to embark on research that mayyield unwelcome results.
A lack of demand for fundamentaland comparative analysis3.8 There may be also a lack of demand forfundamental analysis where, for example,departments have a “sponsor” relationshipwith a particular industry. The research adepartment carries out may depend to asignificant degree on its main stakeholders.
3.9 Sometimes the political process givesMinisters the incentive to set up a range ofgood initiatives rather than to evaluatewhere Government can best focusexpenditure. In such cases comparativeanalysis can be lacking.
Where’s the big picture?3.10 No one department may have sufficientincentive to carry out analytical work wherean issue cuts across several departments. Forexample, Housing Benefit reform involvesthree departments (DSS, DETR and HMTreasury). Environmental issues, such as airquality or global warming, involve evenmore. In these cases, the requirement is toestablish and analyse the “big picture”. Suchdemands are not always met in full.
The supply of analysis also needs improving3.11 Even when demand is present, thereis a range of obstacles to the supply of goodanalysis.
Planning and prioritisation for thelong-term is weak...3.12 Failures in analysis can arise throughshortcomings in planning for the long-term.Long-term work can be crowded out byshort-term priorities.
What are
the problems?
13
...as is planning for cross-cutting issues3.13 There are also problems with planningthe analysis of cross-cutting issues:
• analysis of the trade-offs betweendepartmental goals may not be fullyplanned for or provided;
• gaps in analysis can arise if issues fallbetween the responsibilities of differentGovernment departments and are notplanned for;
• there may be poor central co-ordinationof the work of departments.
Addi
ng It
Up
14
Box 3.1. PENSIM – A Failure of Supply
PENSIM is the DSS model that predicts pensioner incomes over the long term and thereliance of the pensioner population on Income Related Benefits. It is discussed in casestudy 6.
Demand from the Treasury for the outputs from such a model existed over a long period.The Treasury takes a close interest in the impact of pension policy changes on income-related benefits. The outputs from the model were also keenly anticipated as a key inputto the new administration’s review of pensions.
But over a number of years DSS analysts gave higher priority to more urgent work, suchas answering Parliamentary Questions, over trying to make the PENSIM model work.This reflected the day-to-day priorities of their policy customers. And when, more recently,a specialist analyst was recruited to help improve the model he was diverted to work onlong-term care of the elderly: a Ministerial priority of the day.
The result has been a model that did not work at all for several years and which, eventoday, has significant limitations. DSS is now committed to a rolling programme ofimprovements to the model over the next 2 years while a replacement model, PENSIM II,is under development.
Box 3.2. Cross-Cutting Analysis: Trade-Offs, Gaps and Co-ordination
Trade-offs. Competing Government goals where responsibilities span departments include:raising tax revenues while promoting competitiveness; encouraging the efficient use ofwater whilst ensuring no adverse distributional consequences of domestic water chargingpolicies; and promotion of the tourist industry whilst conserving the environment.
A gap: analysis of the socially excluded. A former special adviser told the Project Team ofthe difficulties he had encountered in interesting administrators or analysts in issues aboutyoung unemployed people who are not registered as unemployed – people who are sosocially excluded that they fall outside normal support mechanisms and the usual run ofdepartmental responsibilities. He has since developed and published an influential analysisof this client group based on Government statistics drawn from the Labour Force Survey.The issues are now being taken forward by the Social Exclusion Unit (SEU).
3.14 The interests of the “whole ofGovernment” can also be left out of theequation. Shortcomings in analysis can resultfrom:
• failure to plan or carry out “whole ofGovernment” analysis, e.g. to explore thesynergies between different departments’activities;
• failure to manage systematically theprioritisation of analytical resources acrossGovernment.
3.15 The clearest example of this relates tothe distribution of specialist resources withinWhitehall. Table 3.1 shows the distributionof specialist resource – in the realms ofeconomics, social research, operationalresearch and statistics only – across Whitehalldepartments. The table compares theGovernment’s spending on different policy
areas with the numbers of staff engagedin analytical work in each area.
3.16 This is a partial picture. It does notinclude the work of actuaries in theGovernment Actuary’s department, whichcurrently employs 38 actuaries and onestatistician. Nor does it include the workof other specialists or take account of theeconomic benefits to the UK of analyticalwork, which can be very significant.Nonetheless, it suggests that relatively smallamounts of economic and statistical analysisare devoted to some high expenditure areas.There is no mechanism to confirm whetherthese allocations, which have emerged fromindividual decisions by departments, give theright balance between specialisms orbetween departments or accord the rightoverall level of resource to analysis andmodelling.
What are
the problems?
15
Co-ordination: tax/benefit modelling and data sources. The Inter Governmental TaxBenefit Model (IGOTM) was originally launched collaboratively as an interdepartmentalmodel. But DSS has preferred to rely on its own Policy Simulation Model (PSM) and notto adopt IGOTM. Similarly, difficulties agreeing changes to the Family Expenditure Survey(FES) with other users and with ONS led to a decision by DSS to go it alone and set upthe Family Resources Survey (FRS). The FRS provides a much better basis for modellingDSS policies. Similarly PSM is better geared than IGOTM to DSS priorities. But in bothcases, the outcome involves a degree of overlap and duplication of effort. It was certainlynot consciously chosen by Government as a whole as the best overall way forward.Despite this overlap, efforts have been made by both departments to co-operate.
Box 3.3. “Whole of Government” Analysis and Prioritisation
Electronic delivery of Government services. Government is now undertaking intensivestudy of the synergies between electronic delivery of Government services acrossGovernment as a whole. But for some years, analysis was carried out within departmentsalone. See case study 17.
Prioritisation of data availability. Data on inflation and employment are availableafter a much shorter time-lag than data on poverty. This may to some extent representa Governmental decision about priorities. But in fact the Government has no clearmechanism for determining whether the prioritisation that emerges from departmentaldecisions accords with overall Government priorities.
Addi
ng It
Up
16
Table 3.1: Distribution of economists, statisticians, social researchers and operationalresearchers by department
Operational Social Government Specialists1
Total Economists Researchers Researchers Statisticians Spending
(%) (%)
MoD2 326 5 300 2 19 8.9 18.2
ONS 275 3 102 170 0.1 15.4
DETR 162 64 5 38 55 17.5 9.0
DfEE 147 35 28 42 42 6.6 8.2
DSS 130 35 37 26 32 39.2 7.3
HMT 114 101 1 1 11 0.1 6.4
HO 110 4 17 63 26 2.9 6.1
DTI 101 54 11 36 1.3 5.6
DH 96 25 23 2 46 16.0 5.4
DFID 53 45 8 1.0 3.0
MAFF 51 26 25 1.5 2.8
IR 47 9 17 21 1.3 2.6
C&E 31 7 10 14 0.4 1.7
ECGD 22 9 13 1.2
OFGEM 21 21 1.2
HSE 16 5 11 0.9
CO 15 9 4 2 0.1 0.8
FCO 15 14 1 0.4 0.8
OFT 15 15 0.8
National Savings 10 0 1 7 2 0.1 0.6
LCD 8 1 3 4 1.0 0.4
Competition Commission 7 7 0.4
OFTEL 7 7 0.4
SSRA 7 5 1 1 0.4
OFWAT 6 4 2 0.3
ORR 4 4 0.2
Forestry Commission 4 2 2 0.0 0.2
DCMS 4 2 2 1.4 0.2
Total 1804 518 467 287 532 100.0 100.0
Source: Government Statistical Service Annual Report; Economist Group Management Unit; Government Social Research Annual Report and
Chair of Government Operational Research Service
1 Percentage of specialists employed in Government.2 The number of OR staff shown against MoD is an indicative estimate only, since, within the defence environment, OR merges
imperceptibly into weapon system assessment work which is not directly comparable with anything undertaken in civil Departments.
In total, the Centre for Defence Analysis (a sector of DERA) employs about 500 civilian specialists.
Joint working is patchy; analystsdon’t always have enough influence3.17 Effective joint working betweenanalysts and policy makers is one of thekeys to anticipating the policy agenda, andto making full use of the analytical capacityof Government. But joint working is patchy.Sometimes analysts are seen as mere“number-crunchers”. Within policycommands, little use may be made of smallscale modelling or of spreadsheets or graphsin everyday policy work and in thepresentation of such work to Ministers.
3.18 Measures that may be employed tocounter these problems have not (yet) beenthat widely adopted in Whitehall:
• some departments “bed out” their specialistsinto the policy management line (to greateror lesser degrees) to help promote jointworking. Departments that have beddedout analysts agree that this has improvedjoint working. Other departments believethere may be a price to be paid if it leads(possibly in the longer-term) to the erosionof specialist skills;
• some departments train policymakers inanalytical skills and analysts in the skills ofgeneralists (public policy administration,management etc.). But this is notwidespread in Whitehall.
3.19 The synergies between specialists mayalso be under exploited. There are often closelinks between analysts from differentspecialisms within departments. But theGovernment Economic Service (GES) and theGovernment Statistical Service (GSS) eachhave their own management arrangementsand annual conferences. The GovernmentOperational Research (GORS) andGovernment Social Research communities(GSR) also have their own separate annualconferences. There is scope for moreextensive joint working between specialistsat Government level.
Specialist skills are in short supply3.20 Analysis in central Government willultimately only be as good as the peoplerecruited and retained to do this work(or commissioned to do so throughcontracting-out).
3.21 It is particularly difficult to recruiteconomists at the moment:
• at the Civil Service Selection Board (CSSB),the main entry route to the GES forgraduate economists, it has provedpossible over the last year to recruit onlyaround 75 candidates in response to bidsfrom departments for around 120 newentrants. Demand was distinctly higher lastyear than in the past, but it has neverproved possible to recruit the full numbersbid for by departments;
• departments can recruit short-termprovisionals (casual staff who have notpassed CSSB but may get through a futurecompetition) or turn to direct recruitmentat the higher Grade 7 – middle manager –level. But not all departments meet theirrequirements for direct entrants at thislevel either.
3.22 There are also varying degrees ofdifficulty in the recruitment of otherspecialists. Details are in Chapter 8.
3.23 For all specialists, there are issuesaround lack of career opportunities. Theratio of those at Grade 6 and 7 in the GESto the number of Grade 5 posts (the lowestrung in the Senior Civil Service) has increasedfrom 3.5 to 1 in 1991 to 4.9 to 1 in 1999.Many senior Chief Economist posts (atGrade3/Grade 2 level) have disappeared inrecent years. For social researchers, the postof Head of Profession in a Whitehalldepartment (where this exists) is normally atSCS Grade 5 level. In the case of operationalresearchers, there has also been significantdown grading of posts in recent years. Only
What are
the problems?
17
MoD, DSS and DH now have a Head ofProfession within the Senior Civil Service.
3.24 This in turn places a premium on jobsatisfaction. This can be low when analystsfeel isolated from the mainstream of thepolicy-making process.
3.25 There are, finally, a number of specificskill shortages within the analyticalcommunity as a whole:
• in-depth statistical understanding.More sophisticated data require richerand deeper numerical and statisticalunderstanding on the part of the user.An Economic and Social Research Council(ESRC) working paper on social statisticssuggests that these skills are not widelyavailable even within the academiccommunity;
• IT skill shortages. Developments incomputing power have pushed back thelimits of the possible in analysis by offeringmore rapid ways of manipulating largedata sets. For example, a single run of theDSS PENSIM model once took 8 hours tocomplete but can now be carried out inaround 20 minutes. Computing poweris not a limiting factor on analyticaldevelopment, but does imply the needfor greater sophistication by modellers;
• keeping up to date. Analysis need not beground breaking to be an effective inputto policy. But there is a constant need torenew professional skills. This is not alwayseasy for analysts outside academia.
More data, better used...3.26 A vast array of data is available to policymakers. But this is not always the data thatare most needed. Two particularly valuableforms of data which are often absent – orunused – are those derived from pilots andlongitudinal studies.
Box 3.4. The Government’sUse of Pilots
Pilots allow evidence of the effects ofa policy change to be tested againsta genuine counterfactual (such as isprovided by the use of control groups ina medical trial). This gives the strongestpossible evidence of what works.
The Government trials many policiesbefore full implementation, but oftenas a way of exploring the practicalitiesof implementing policy rather than asa means of gaining early data aboutthe impacts of policy interventions.
Often Ministers will announce a pilotat the same time as dates for nationalimplementation of a policy. This limitsthe timeframe within which lessons canbe learned. Even so, more thought couldbe given to the design and evaluation oftrials so that they can be used as pilots toimprove policy prior to national roll-out.
Addi
ng It
Up
18
3.27 There are also shortages of data relatingto small geographical areas. And sample sizesin surveys may be too small to assess theeffects of policy on subgroups of thepopulation to the extent that Ministersrequire.
Box 3.6. Difficulties in CollectingSmall Area Data
Sometimes a statistically significantsample at the local level would be tooexpensive to collect. In the case of theBritish Crime Survey, a survey whichgenerated useful data at the level ofpolice forces would cost £14 million pa(by comparison with a £1.5 pa cost foran annual national survey). But thiswould not yield information that wasuseful at police command level or forsmall geographical areas.
Box 3.5. Longitudinal Studies
Longitudinal studies trace the life-historiesof individuals across time. They enableunobserved differences between individualsto be accounted for in analysis andcausal links to be established.
The UK is a world leader in birth cohortstudies. In some ways this is fortuitous.Many studies created for a particulartime-limited purpose have in the eventbeen extended. But sometimes fundingfor extensions of surveys has beendifficult to obtain. And in otherlongitudinal studies, e.g. of ageing, theUK lags behind countries such as the US.
There is scope to make more use oflongitudinal data. Use within Governmentmay be limited in part because the datasets are complicated to work with – soresearch and analysis tend to becontracted out to the private sector.
3.28 Some kinds of information (e.g. incomedata) are not sought in surveys because ofpossible adverse effects on the response rate.This happens, for example, in both theCensus and British Social Attitudes Survey.The absence of data of this kind in surveycan impede analysis, e.g. of the distributionalconsequences of a policy reform. There isa technical solution to some gaps (thetechnique of “imputation” discussed inChapter 9) but it is little used indepartments.
3.29 There is also scope to make more use ofadministrative data. Even where Governmenthas extensive administrative data sources,these are often not widely known within oroutside Government. Ignorance about DSSadministrative records is an example of this –though this is an area where confidentialitycan be a barrier to making data more widelyavailable.
...better planned and of better quality3.30 The GSS carries out a central co-ordinating role on data issues through itscommittee network. These arrangements arewidely appreciated by statisticians working indepartments. But difficulties haveoccasionally arisen, raising two questions:
• whether the central co-ordinating bodyneeds more authority to impose a solutionon departments that has benefits for thewhole of Government but disadvantagesfor individual departments;
• whether decisions should be taken by abody of users of statistics or by GSS aschief suppliers.
What are
the problems?
19
3.31 Finally, there are sometimes doubtsabout the quality of recorded data. Policeadministrative data (the basis for the HomeOffice property crime model), for example,may exaggerate clear-up rates. Survey dataalso have pitfalls. People may not understandfully their financial affairs (e.g. their pensionentitlements) and misreport their position toresearchers. The running order of questionsin a survey may influence the repliesrecorded. For example, a change in therunning order of the Labour Force Survey(LFS) in 1997 resulted in a 25% change inthe response to one question on disabilities.
Box 3.7. Co-ordination BetweenSurveys...
The number of surveys has grownrapidly in recent years as new surveyshave been established to meet the policyagenda of the day. It is not clear that thepresent set of surveys gives idealcoverage.
...and Uniformity of Definitions
ONS has worked to establish uniformityin definitions of key concepts used insurveys run by departments (e.g. what isa “household”). It has also examined theextent to which it may be possible toobtain more uniformity acrossadministrative data sets. But informationis still not always stored by departmentsin a uniform manner – whether thisrelates to survey data, data fromadministrative systems or details ofspending. Not all departments were, forexample, able to provide details ofspending on 0-3 year olds to the SureStart programme. Administrative datacollected by local authorities are also,perhaps almost inevitably, not allcollected or stored in a uniform way.
It is important that possible limitations areclearly understood by all users of data.
Models don’t get out enough....3.32 Where models are a necessary part ofpolicy analysis, there is a risk that they takeon lives of their own, gathering a priesthoodaround them for their defence against anuncomprehending external world of policymakers within the department, other analystsin Whitehall or outsiders.
3.33 The extent to which external review issought for models is patchy. Most modellersin Whitehall do actively seek comment ontheir models and review by colleagues. Butthe position varies between models andarrangements are informal and ad hoc. Andthere is a general unwillingness amongstdepartments to make the running of modelsmore open to others inside or outsideWhitehall.
...and modelling needs to be better planned and of higher quality3.34 The priority accorded to models variesover time. For example, the Treasury’sIGOTM model has in recent years had moreresources devoted to it. Some models, suchas PENSIM and National Road TrafficForecasting model (NRTF), have longgestation periods. Consistent investment overseveral years is required to achieve results. Akey issue is the extent to which longer-termwork is squeezed out by the short-term.
3.35 The big picture is not always modelled.While individual departments may find theirown modelling needs are adequatelyaddressed, the totality of the policy area maynot be captured.
Addi
ng It
Up
20
3.36 In principle, there is a case for the“big picture” to be modelled by a single,central department. The case study of theGovernment Actuary’s Department (GAD)population model (see Box 3.8 and casestudy 4) suggests that sometimes it is indeedhelpful for a single central department to“hold the ring”. But the tendency in recentyears has been for individual spendingdepartments to develop their own modellingcapability. For example, DSS have taken overmodelling of some disability benefits fromGAD. And upkeep of IGOTM, which was fora number of years run by ONS on behalf ofthe Treasury and IR, is now undertaken in-house by the Treasury.
3.37 Where more than one department isinvolved in modelling, failure can take theform of competing analyses as well asabsence of analysis. For example, whenpolicy options were being developed forsplitting pension entitlements betweenmarried couples on divorce, it took the LordChancellor’s Department (LCD) and DSS thebest part of a year to agree what the impactwould be on legal aid bills.
Box 3.8. Population Modelling
The Government Actuary’s Departmenthas, since 1954, provided populationprojections for the whole of Government.In this case, there are clear advantagesto Government in the existence of asingle central model. There is no risk ofwork on this model being crowded outby more urgent departmental concerns.And all departments are constrained touse a single set of figures for populationin their own policy work. The inevitableconsequence of such an arrangement isthat not all Whitehall customers arealways content with the assumptionsused in the GAD model, or with itsoutputs.
3.38 Finally, models often can’t take accountof behaviour. This problem is highlighted inthe projection of benefit spending.Departments (DETR, the Treasury and DSS)can calculate the cost of policy proposals ifthey lead to no behavioural change. But thewhole point of most reform is to changebehaviour. The Government does not knowhow much policy options will cost or saveif they work.
3.39 Chapters 5-11 discuss ways to addressthe problems set out in this Chapter. Thenext Chapter offers a vision of the role ofanalysis and modelling in centralGovernment in an ideal world.
What are
the problems?
21
4.1 The purpose of this Chapter is to setout a vision for analysis and modelling incentral Government and the conditionsneeded to achieve it. It is informed by wideranging discussions inside and outsideWhitehall, and in particular by a visit to theUnited States (see box 4.1) which providesa benchmark for best practice.
In an ideal world...4.2 In an ideal world, we would expectto see:
• a vigorous demand for good analysis fullyembedded in Government. Ministerswould routinely demand rigorous analysisto support decisions on all policies andprogrammes. Senior civil servants wouldsee it as a key aspect of their job to ensurea reliable supply of such analysis;
• a culture in Whitehall that rewards Ministersand senior officials for good analysis and holdsthem to account when policy is not evidencebased. Ministers would publish details ofthe models used in government and thedata that support them. This would helpfoster an external climate in which, as in theUS, academics and think tanks competewith analysts in Government in all forms ofanalytical work and improve its overall quality;
• planning mechanisms to ensure the supplyof good analysis where demand is likelyto be weak – e.g. when issues are cross-cutting or long-term. Responsibility forthe provision of analysis would rest withdepartmental Accounting Officers. Butan enhanced central challenge rolewould help ensure sound planning andprioritisation. Heads of Profession wouldalso have a key role to play in this process.
4. ANALYSIS IN AN IDEAL WORLD
Summary
This Chapter sets out a vision of analysis and modelling in centralGovernment inspired by best international practice. Key elements are:
• vigorous demand for good analysis supplemented by strong planningmechanisms for long-term and cross-cutting issues;
• closer working between policy-makers and analysts;
• good and plentiful data on tap;
• better models open to external scrutiny;
• analysts of the highest calibre.
Addi
ng It
Up
22
4.3 A number of elements would needto be in place to support this vision.
Policy makers and analysts working closely together4.4 Soundly based analysis can only besupplied by economists, actuaries,statisticians, social researchers andoperational researchers if there is:
• a responsive capacity for analysing newand existing initiatives;
• a shared understanding between policymakers and analysts across departmentsabout what is required;
• an agreed analytical and modellingstrategy in each department as part of thebusiness planning process, with researchcommissioned early in the policyformulation process;
• an understanding by policy makers of theuses and limitations of analysis and modelsin the policy formulation process.
Good and plentiful data on tap4.5 In an ideal world high quality datawould be readily accessible and used acrossWhitehall. More specifically there would be:
• no obvious data gaps;
• routine use of data description to assistboth policy making and decision-taking;
• widespread availability and use of largehousehold surveys across departments;
• routine use of administrative data acrossWhitehall;
• more creative use of data and differentdata sources;
• use of panel and birth cohort surveys toshow the long term impacts of policy;
• more use of pilots to help shape policydesign before national implementation.
Better and more open modelling4.6 In the case of modelling, there wouldideally be:
• models in place in all appropriate policyareas;
• models which were published, debatedand regularly audited;
• models which were shared acrossdepartments and which were capable ofdemonstrating the trade-offs betweenconflicting objectives.
The right skills4.7 Ideally there would be:
• recruitment and retention of both thebest, and the right mix of, specialistswithin the Civil Service;
• the best use of external expertise;
• more flexible structures in place for thosewho would add value but do notnecessarily have the attributes to reach theSenior Civil Service (one of the key criteriafor recruitment to e.g. the GES).
The United States providesa benchmark4.8 The United States is consistently citedas a world leader in analysis and modelling.Members of the project team visitedWashington with a view to learning widerlessons. The team found that the US wassome way ahead of the UK both in thegeneral thirst for analysis in Government andin the use of data and modelling. A summaryof the findings is in Box 4.1 and a full reportis at Annex 4.
Analysis in
an ideal
world
23
Addi
ng It
Up
24
Box 4.1 The US – Higher Quality Data and Debate
Some of the reasons for US superiority in analysis, notably the nature of the US constitutionwhich pits administration against Senate and generates a competitive interest in modelling,are not directly transferable to the UK. But four fundamental lessons are of widerapplicability:
• a strong willingness to invest in gathering the necessary data. The culture of thoroughprior analysis and ex post evaluation means that data are valued highly in the US andresources are devoted to its collection;
• imaginative and widespread use of administrative data. Although faced with similarconfidentiality obstacles to the UK there appears to be a greater willingness on the partof the US Government to make available administrative data, once they are suitablyanonymised, for use within and outside Government;
• outside expertise is transmitted easily to Government. Relations between Governmentand the academic sector are much closer than they typically are in the UK;
• a richer environment for analytical debate. The volume of analysis inside and outsideGovernment creates an assumption amongst analysts that excellence is necessary forcredibility. When supported by better data sources this makes for more technicallyadvanced modelling.
5.1 The purpose of this chapter is to reviewthe means by which the demand for goodanalysis can be increased. The package ofmeasures proposed should bring about aprofound change to the culture in whichcivil servants operate. The key ingredientsin such a package are:
• improved training and best practiceprotocols;
• increased openness;
• increased use of peer review;
• improved financial incentives for cross-cutting and long-term analysis;
• increased emphasis on analysis inevaluation processes.
Ministers and senior officials must want good analysis5.2 Ministers and senior officials can makea real difference to the quality and quantityof analysis supplied in individualdepartments.
5. STIMULATING THE DEMAND FOR GOOD ANALYSIS
Summary
The demand for good analysis can be increased by:
• stronger leadership from Ministers and senior officials, stimulatedby training and dissemination of best practice protocols;
• increased openness;
• peer review of departments’ analytical strategies, with a centralTreasury team drawing together the “big picture” on the economicand social agenda;
• improved financial incentives for cross-cutting analysis throughthe setting up of a seed-corn fund;
• increased emphasis on the quality of ex ante analysis and ex postevaluation of policy based on “Green Book” principles.
Stimulating the
demand for
good analysis
25
Raising expectations throughtraining and sharing best practice5.3 The CMPS runs training courses fornew Ministers and is planning joint trainingfor Ministers and senior officials. Thesecourses provide an opportunity to:
• emphasise the importance of leadershipfrom Ministers and senior officials in raisingstandards of analysis;
• demonstrate how analysis and modellingcan improve policy development anddecision-making.
5.4 Case studies might be used to illustratethe usefulness of good analysis. For example:
• rigorous analysis can help Government settargets for public services that arechallenging but capable of delivery;
Box 5.1. Ministers Can Make aDifference: The “Castle Effect”
Significant differences in departmentalculture and practice can be observedacross Whitehall. In some cases thisreflects a conscious effort to increase theprofile of specialists or particular kinds ofspecialist expertise in individualdepartments. For example, Barbara Castlewas responsible in the 1960s for theintroduction of significant numbers ofeconomists into the (then) Ministry forOverseas Development and later the(then) Department of Transport. Theseeconomists took strong root in bothdepartments: their descendants are todaywell embedded in the policy process inboth departments (now DFID and DETR).
• a “full systems” analysis which identifiesand analyses likely behavioural impactsof policy options may improve decision-taking. For example, Ministers would wantto be aware of the extent to which dutyincreases against a backdrop of open fiscalborders would lead to increased incentivesfor cross-border shopping and smuggling;
• good analysis can bring benefitsinternationally as well as domestically. Forexample, work on the balance of costs andbenefits in setting domestic air qualitytargets has enabled Ministers to influencethe development of EU and internationalpolicy on air quality targets.
Conclusion 1: training courses for Ministers and senior officials run by the CMPS shouldin future include a session on the importanceof good analysis for the policy-making anddecision-making processes.
5.5 CMPS will also be responsible for thedissemination of policy making best practiceacross Government. This will supportevidence-based policy. It is important thatguidance include advice on data andmodelling issues. For example, Chapter 9concludes that CMPS should establish anddisseminate best practice on the use oflongitudinal studies and pilots in policydevelopment.
5.6 There is already best practice guidancefor technical experts in Governmentdepartments about how to carry out ex anteappraisal and ex post evaluation of policy:the Treasury Green Book (5).
5.7 The Green Book is presently beingrevised. As currently drafted it is directedprimarily at economists. But best practiceprotocols issued by CMPS might includeworked examples of best practice in theconduct of appraisal and evaluation andbe issued to a much wider audience: toMinisters and to all policy makers and
Addi
ng It
Up
26
professionals across Whitehall to helpdemonstrate the practical application ofGreen Book principles.
5.8 A further ready-made source of bestpractice advice lies in the value for money(vfm) studies of the National Audit Office(NAO) (considered in more detail later in thisChapter). There is scope for more widespreaddissemination of NAO findings. The NAO isproposing to make available its reports on itsweb site. There may also be scope for CMPSto help with dissemination and for NAOreports to become an integral part of theknowledge pool that helps guide policydevelopment.
Conclusion 2: CMPS should include guidance on the use of analysis in bestpractice protocols and circulate it to Ministersand officials. Such guidance should helpdisseminate “Green Book” principles innon-technical terms (e.g. through the use ofworked examples) as well as the findings ofNAO vfm studies.
Being open improves the quality of analysis5.9 Transparency promotes the demand forgood analysis. The more that Government isobliged – or obliges itself – to defend policydecisions by publishing the underlying dataand analysis, the more emphasis theGovernment will place on getting analysisright. Freedom of Information legislation willincrease transparency in Government byproviding new statutory rights of access toinformation.
5.10 Government can, in the extreme,legislate to require open disclosure ofanalysis. In the US, the General Audit Officeis required by legislation to provide a cost-benefit analysis and a distributional impactassessment of any proposal that will costmore than $100 million pa.
5.11 Less formal Government mechanismscan also help to ensure that rigorous analysisis carried out. There is a requirement topublish a Regulatory Impact Assessment (RIA)with all new legislation which sets outimpacts on business. The RIA must also beproduced by departments as a condition ofCabinet endorsement of any policy. Similarrequirements to publish impact assessmentsapply in relation to the environment, health,gender, race and older people. Suchrequirements are not legislative, but havebeen agreed by Cabinet Committees.
5.12 These mechanisms stimulate demandfor analysis of the impact of policy options.Their force derives in part from the simplerequirement that such analysis is carried outand in part from the requirement to makepublic the results of the analysis.
5.13 There is an initiative underway withinthe Cabinet Office to align the various impactrequirements. A pilot study is beingconducted in DETR. Subject to the outcomeof the current review, there is a strong casefor requiring departments to carry out andpublish a single comprehensive impact
Box 5.2. Legislating forOpenness in the UK
In the UK legislation dating from themid 1970s requires the GovernmentActuary (GA) to report on the level ofthe contracted out rebate (for pensionschemes and, more recently, individualscontracted out of SERPS) at least once inevery five years. Responsibility for settingthe rebate rests with Ministers, but theGA’s report is published. This assuresemployers and pensions industryanalysts of the Government’s good faithin setting a level of rebate that isunderpinned by rigorous analysis.
Stimulating the
demand for
good analysis
27
assessment. This should cover all the relevantimpacts and the trade-offs between them.
Conclusion 3: subject to the outcome of the present review, departments should carryout and publish a single comprehensiveimpact assessment of all new policies,programmes and projects.
5.14 A significant step towards greateropenness for statistics is proposed in theWhite Paper Building Trust in Statistics (4).The intention is that a National Statisticianshould develop and publish a forwardprogramme of statistics covering the wholeof Government. This would then be subjectto published evaluation and comment froman independent Statistics Commission.
5.15 There is a case for extending thisapproach to cover the full range of researchand analysis within Government. This wouldprovide a vehicle for eliminating overlaps andspreading information across departmentalboundaries. This task sits best with theexisting functions of the Central OperationalResearch and Economics (CORE) commandin HM Treasury.
5.16 In the first instance, such an overviewmight be confined to the economic andsocial agenda. The CORE team might presenta set of rolling thematic reports to either theproposed Academic Panel for Analysis andModelling (see Chapter 10) or to theImplementation Group charged withoverseeing the implementations of this report(see Chapter 12). A decision could be takenat a later date whether to extend this acrossthe entire field of Government research andanalysis.
Conclusion 4: the CORE team in HM Treasury should review annually departmentalplans for analysis and research and present areport setting out the main findings to theImplementation Group charged withoverseeing the implementation of this report.
Peer review can help spread best practice5.17 A regular published audit of departments’programmes of analytical work would serveas a further incentive to departments to carryout high quality analysis. This could formpart of the independent peer review ofdepartments’ business planning processesproposed in the context of the reform of theCivil Service. The review team would needto include suitable expertise, possibly fromwithin CMPS or Treasury spending teamsor outside the Civil Service.
5.18 In addition, CMPS has a remit todevelop a broader programme of peerreviews across departments, includingexamination of policy-making processes.Those are likely to include a look at the useor absence of data, the quality of analysis,the use of professionals, relationshipsbetween them and generalists and opennessto external knowledge.
Conclusion 5: CMPS-led peer reviews of business planning and policy-making shouldexamine the quality of analysis and modellingwith a view to spreading best practice andstimulating improvement.
Financial incentives can strengthen analysis5.19 The establishment of a small seed-cornfund for analytical work would also helpstimulate the demand for good analysis.
5.20 In some instances no one departmenthas sufficient incentive to do work that isclearly justified in terms of its overall benefitsto Government and is a public good. In otherinstances, the pay offs are long term anddepartments are unwilling or unable to investto secure the resulting benefits.
Addi
ng It
Up
28
5.21 The case for a central fund is clearwhere the requirement for analytical workis cross-cutting. Where the requirement foranalysis sits firmly within the remit of a singledepartment, the standard mechanism toensure that this work takes place should bethe Spending Review and Public ServiceAgreement processes supplemented byimproved business planning in departments.However, in the case of long-term research,analysis or model development, requiringunusually substantial amounts of initialfunding, there may be a case for using thefund to intervene. It would be important fordepartments to be able to demonstrate thatcentral funding would not lead to any“crowding out” of work that would havebeen undertaken anyway.
5.22 The CORE team in the Treasury is thenatural home for the administration of sucha seed-corn fund. It has the best overviewof the work of individual departmentsthrough the Treasury expenditure teams.It is important, however, that any committeeto disburse funding should includerepresentation from the Cabinet Office andanalysts (of different specialisms) from otherdepartments. Additionally, the CORE teammight itself draw on secondees fromdepartments or from the private sector.
5.23 Bids for funding should be welcomedfor analytical work in any specialist discipline.The fund will work most effectively ifdepartments are required to match or morethan match any central funding. This shouldhelp instil a sense of ownership for suchresearch beyond the centre.
Stimulating the
demand for
good analysis
29
Box 5.3. Work That a Seed-Corn Fund Might Promote
Examples of the work a seed-corn fund might help include:
• further longitudinal studies: the lack of longitudinal data gives the UK much poorermodelling of long-term care of the elderly than the US. But longitudinal studies are anexpensive and long-term commitment. Their full value, across Government as a whole,is often clearer to those in the centre than to individual departments. But the centre hasno research funding and cannot at present lead or even contribute towards a fundingconsortium to launch new longitudinal cross-cutting surveys;
• developing a multi-modal national transport model: the costs of developing sucha model within DETR would be large and the opportunity costs would be larger. Sucha project would exclude most other items from the DETR work programme. In such acase, the high upfront costs will always militate against such work being carried out. Inthis case, DETR have, in the event, identified ways of meeting most of the policy analysisrequirements that such a model would address, building on the National Road TrafficForecasting Model.
Conclusion 6: a seed-corn analysis and modelling fund should be established fromthe financial year 2000/2001. This would besubject to a successful bid on the Reserve for2000/2001 and in the present SpendingReview for future years. The fund should beadministered by the Treasury with bidsconsidered by a committee drawn fromdifferent specialisms and different departments.
Evaluate to learn5.24 The ex ante development of policies,programmes and projects can benefit fromthe results of ex post evaluation studiescarried out within departments.
Addi
ng It
Up
30
Box 5.4. How Much Will a Seed-Corn Fund Cost?
To provide an indication of the resources that might be required to establish a seed-cornfund, the project team has analysed the kinds of improvements that can be made for givensums of money. The indicative costings are based on discussions with the project’s SteeringGroup members, information on costs of existing models, and external consultants.
• Construction of a new household formation model. The present model used by DETR iscontracted out through the Building Research Establishment to Anglia PolytechnicUniversity at a cost of £150,000 per year. A project of 6 person months (the minimumrequired to build a new model from scratch) would cost around £200,000 if conductedby external consultants.
• A new PENSIM pensions model – the original model (commissioned in January 1988)cost around £60,000-£70,000. Other countries – notably Canada and Australia – havespent around ten times that amount. DSS has indicated that the new PENSIM will largelybe developed in house and is expected to take up around 1-2 years of Grade 7 time.
• The London School of Economics (LSE) is currently developing a model coveringlow-income pensioners and long term care of the elderly with a budget of £0.5 millionover a period of 5 years.
• The economic assessment of the costs and benefits of different options for the electronicdelivery of Government services. A three-month project to provide some baselineestimates might cost in the region of £50,000-£100,000.
• Construction of a model to show the distributional effects of different methods ofcharging for domestic water services – a model provided by outside consultants forDETR in 1998 cost around £30,000 for three months’ work.
The seed-corn fund might also provide departments with help for the funding of datacollection. Examples of the type of costs that might be involved include:
• a module of 40 questions in the British Social Attitudes Survey costs £40,000 for 1/3of the sample (1,100 respondents), £42,400 for 2/3 of the sample and £53,000 forthe whole sample. That is substantially cheaper than a stand-alone survey would be;
• the Census rehearsal currently being undertaken at a cost of £2 million covering147,600 households across the United Kingdom.
These examples suggest that the appropriate magnitude for a seed-corn fund wouldbe around £3-4 million pounds pa.
5.25 The National Audit Office (NAO)examines the economy, efficiency andeffectiveness of departmental administration.Departments can learn lessons for the futurefrom NAO value for money (vfm) studies.They can also increase incentives for the useof rigorous analysis. Indeed, such studies maythemselves include consideration of theextent to which decisions have beendemonstrably informed by good analysis and,where appropriate, modelling. This was afeature of a report on the sale of railfreightdistribution (see Box 5.6).
Box 5.6. The Sale of RailfreightDistribution
The NAO study of the sale of railfreightdistribution found that DETR relied onpoint estimates of the relative costs ofprivatisation, retention in the publicsector and closure supported by asubjective assessment of risk. DETR didnot conduct any quantified risk analysis.NAO believed that a quantified riskassessment would have helped to givegreater confidence in the DETR decisionto privatise and therefore commissionedsuch a risk analysis from externalconsultants. In the event, this suggestedthat in the light of the possible range ofcosts, the decision to privatise was sound.
Stimulating the
demand for
good analysis
31
Box 5.5. Learning the Lessons from Policy Evaluation
Within a continuous cycle of policy development, evaluation is a key input to the appraisalof new policies. Evaluation of past policies enables the design and delivery of current andfuture policies to be improved in the light of experience.
A Treasury initiated review of evaluation within DoE in 1997 concluded that the use ofsystematic practices across the department was patchy. A more recent Cabinet Office studylooking at evaluation across Whitehall reached similar conclusions. But the report foundthat there have also been a number of high profile policy initiatives which have been thesubject of careful monitoring and evaluation, conducted in a context which is tuned in wellto learning lessons and applying the results of pilot work. Examples included Pathfindersfor the New Deal for Young People. In this case, three evaluation reports in December1998 identified gaps in the provision for the most marginalised and disadvantaged, withsubsequent corrections to remedy those deficiencies.
And in some departments the lessons from evaluation are valued and systematicallyembedded into the policy formulation process. For example, DfID has a detailed set ofoffice instructions, used by specialists and administrators alike, which sets out the role ofevaluation within project cycle management. This guidance describes how evaluation canbe integrated into the process of project management, from the point of drawing up theproject proposal, through to implementation and monitoring to the point of completionof the project.
Judges and Accounting Officers can help promote analysis5.26 The threat of Judicial Review andthe system of Ministerial Directions toAccounting Officers also provide incentivesfor sound analysis.
5.27 Judicial Review has risen in prominencein recent years. This trend is likely to increasefurther as the incorporation of EuropeanHuman Rights legislation into domestic lawwidens the interpretation of the grounds onwhich review can be sought. There are threepossible grounds for Judicial Review:
• illegality: that the executive does not havethe power to act;
• irregularity: that the executive failed tofollow proper procedures;
• irrationality: that the executive did notreach the decision in a thorough enoughway.
5.28 Soundly based analysis, as long as itis given proper consideration in the policyprocess, is a bulwark against Judicial Reviewunder the last of these headings. Courts mayreview the processes by which decisions arereached and will rule against a decision if it isdeemed that no one, sensibly advised, couldhave come to such a decision.
5.29 Accounting Officers (usually thePermanent Secretary or the Chief Executiveof an Executive Agency) have a responsibilityto ensure that full analysis is suppliedwhenever a decision is taken that hasfinancial implications. Where a Ministercontemplates an action which involves atransaction which an Accounting Officerconsiders would infringe the requirements ofpropriety or the Accounting Officer’s widerresponsibilities for economy, efficiency andeffectiveness, it is the duty of the AccountingOfficer to notify the Minister of his or herdissenting advice. If the Minister overrulesthis advice, the Accounting Officer must seekan instruction – a “Ministerial Direction” –from the Minister to proceed with the courseof action in question.
Addi
ng It
Up
32
Box 5.7. The Pergau Dam
Both Judicial Review and a Ministerial Direction were involved in the case of the PergauDam, which offers a good example of how these processes can bolster the importance ofeconomic analysis in policy development. In that instance, analysis by the (then) OverseasDevelopment Administration (ODA) showed that the project would offer poor value formoney. The (then) Secretary of State for Foreign Affairs decided to go ahead with theproject on wider grounds and gave a Direction to the ODA Accounting Officer to proceed.
NGOs sought a Judicial Review of the decision. The Courts ruled that to spend aid fundson the Pergau Dam was illegal when the ODA Act of 1980 and ODA appraisal guidelinesmade clear that the primary object of aid spending was further development of thecountry and that projects had to offer vfm to fulfill this objective.
This was a watershed. Since then, Ministerial Directions which have instructed AccountingOfficers to pursue public spending on wider grounds have been made public throughParliamentary Questions and in response to press enquiries.
5.30 The Directions given over the years 1997-1998 are listed in the Table 5.1.
Stimulating the
demand for
good analysis
33
Table 5.1 Ministerial Directions 1997 and 1998
Date Case Dept
1997 Channel Tunnel Rail Link: scheme of redress for those who DETRexperienced exceptional hardship as a result of blight from the Channel Tunnel Rail Link
1997 Millennium Exhibition Expenditure to regenerate the Greenwich DETRPeninsula and to provide a site for the Millennium Exhibition
1998 Benefits Agency/Post Office Counters Limited (BA/POCL) DSSAutomation project
1998 Secondment: remuneration arrangements for seconded official Northern Ireland
1998 Cadet Property Moffat: Sale of Territorial Army Volunteer Reserve MoDAssociation Cadet Property to Moffat Childcare (Community Group)
1998 Review of Disability Living Allowance under the Benefits Integrity DSSProject – handling overpayments.
6.1 The conclusions in Chapter 5 shouldhelp to stimulate the demand for analysis.But demand for analysis is not enough.Planning is required to ensure that analysis issupplied when it is needed even if demandmay be weak, particularly in relation tolong-term and cross-cutting issues.
6.2 The proposals in this chapter aredesigned to ensure that:
• the right priorities for analysis are setbetween departments and policy areas;
• professional resources are efficiently andeffectively deployed across policy areaswithin and between departments;
• cross-cutting analysis is carried out wherenecessary.
6.3 In developing detailed proposals,it is important to recognise that:
• departmental Accounting Officers shouldretain ultimate responsibility for analysisand modelling work undertaken within thedepartment;
• the PIU report Wiring It Up (3) establishesa presumption that central intervention inthe everyday work of departments isjustified only where joining-up is difficultand the issues are important;
• any recommendations need to fit with thewider Civil Service reform plans underdiscussion by Permanent Secretaries.
6. PLANNING AND IMPROVING THE SUPPLY OF GOOD ANALYSIS
Summary
The Spending Review process and Public Service Agreement targetsshould be used to provide the framework within which the analyticalneeds of Government and individual departments are planned. TheTreasury should provide and implement guidance in this area.
The Treasury and Cabinet Office need sufficient dedicated analyticalcapacity to provide analytical support to ‘whole of Government’ policymaking and to facilitate cross-departmental working where necessary.
An analytical strategy should be an integral part of departmental businessplanning. Specialists and administrators should plan departments’analytical requirements together.
Addi
ng It
Up
34
Public sector leaders must plan for the long term6.4 Public sector leaders are responsible forthe supply of good analysis. This requiresplanning, particularly where demand is likelyto be weak.
6.5 The two key strands to successfulplanning at Governmental level are:
• a clear path for future policy development.This could take the form of the electionmanifesto at the start of an administrationor of the outcome of a Spending Review inmid-term;
• a response to the policy agenda from theanalytical community (in conjunction withpolicy colleagues). The Heads of Profession– Head of GES, National Statistician,Government Actuary and the Chairs ofManagement Groups for operational andsocial research – have a valuable role toplay in confirming that the Governmentagenda is being fully covered by analysisand in safeguarding professional standards.
Leveraging Spending Reviews and Public Service Agreements6.6 The key mechanisms for settingpriorities for policy work betweendepartments, at a high level, are theSpending Review (SR) and the Public ServiceAgreements (PSAs). The SR determines thelevel of funding each department will receiveover the next three financial years. The PSAtargets set out the improvements to publicservices which the Government will deliverover the period covered by the financialsettlement.
6.7 The SR process already supports cross-cutting work:
• the 1997-98 Comprehensive SpendingReview featured a cross-cutting reviewof spending on children, which gave riseto the Sure Start Initiative (cited as anexample of excellent policy-making in theCabinet Office report on ProfessionalPolicy Making (2));
• the next SR will review a greater numberof cross-cutting topics and will encouragecross-cutting analysis to be supplied (andco-ordinated) in support of the review.
6.8 But there is scope for the process to givefurther support to analysis by:
• issuing guidance to departments on theanalytical work which should underpinthe SR and PSAs;
• publishing analytical work underpinningPSAs. Case study 16 on the developmentof the first round of PSA targets in DCMSand FCO found that they had notinvariably been well-supported by analysis.This is hardly surprising given the severetime constraints under which initial PSAswere developed. But a commitment topublishing future analytical work wouldincrease the incentives for good analysis.
6.9 Not all government objectives,however, will be covered by PSA targets; andnot every cross-cutting issue can be reviewedin a single Spending Review. And resourcesmay need to be allocated to policy areaswhere analysis is presently lacking. So theSR and PSA processes alone will not deliverall that is required by way of long-term orcross-cutting planning. Other mechanismsare needed to supplement these processes –notably an extension of the challenge roleof central departments and better businessplanning within departments.
Planning and
improving the
supply of
good analysis
35
Conclusion 7: the Treasury should use the SR and PSA processes to support andpromote the use of analysis and modellingthrough: (i) issuing guidance to departmentsand (ii) publication of analytical workunderpinning PSA targets.
Stretch, link and learn:the role of the centre6.10 Central departments should play a widerange of roles in supporting the demand andsupply of good analysis. Table 6.1 gives asynopsis of the conclusions in this report.
6.11 In the planning context, the challengefunction is particularly important.
6.12 Much of the everyday work of Treasuryexpenditure teams involves the shadowingand monitoring of policy and expenditureby departments. The Cabinet Office performsa similar function, but over a narrow rangeof areas. Both departments can play animportant challenge function. This can helpensure that analytical work is well plannedby departments, particularly where issuesare cross-cutting.
Addi
ng It
Up
36
Table 6.1: The Role of the Centre
Role Central Conclusions CommentsUnit
Facilitation CMPS Training for generalists Conclusion for departments; (conclusion 10) CMPS interest as supplier
CMPS Knowledge management/ Core aspect of CMPS rolespreading best practice(conclusion 2)
CMPS Peer review of departmental Extension of civil service business planning (conclusion 5) reform proposals
HMT Seedcorn fund for analysis New proposal(conclusion 6)
Co-ordination HMT Modelling Panel to oversee New proposalpublication of models (conclusion 34)
HMT Cross-cutting look at departmental Builds on proposals for analytical strategies (conclusion 4) National Statistics
ONS Data co-ordination Extension of existing role(conclusions 29-32)
Challenge HMT/ More specialists in the centre Extension of existing roleCab Off (conclusion 8)
6.13 However, to carry out the challengerole effectively, both departments may needto expand their analytical capacity.
6.14 The centre does not have a monopolyon the co-ordination of cross-cutting policyand analytical work. But it does generallyneed to be involved. There are a number ofmechanisms in Whitehall today, each ofwhich requires a degree of centralinvolvement:
Box 6.1. The Treasury CanPrompt SuccessfulInterdepartmental Working
There are now no fewer than four inter-departmental groups on costs andbenefits in which DETR plays a part. Two(one on climate change, one on airquality) have been established in recentyears in part as a response to pressurefrom the Treasury to ensure a full andproper cost/benefit analysis ofenvironmental policy measures.
• lead departments can successfully facilitateco-ordination. Work on analysis to supportpolicy on climate change and air quality,for example, is presently led from DETR.Indeed, in general, this method of co-ordination should be the preferredapproach. Case study 15 on air qualitysuggests that it works well, up to a point,but still needs a Cabinet Committee toresolve interdepartmental disputes. Thecentral department may then require theanalytical capability to understandcomplex analytical issues;
• sometimes central units (SEU and WU inCabinet Office) have led interdepartmentalgroups to take forward cross-cutting policyissues and the associated analysis;
• in other cases a central unit such as theRegulatory Impact Unit can give additionalmomentum to a central requirement,while the lead responsibility remains withdepartments;
• the creation of a central knowledge pool –proposals for which are presently beingdeveloped by CMPS – is a role best takenforward by the centre but working closelywith departments.
Planning and
improving the
supply of
good analysis
37
Box 6.2. Analysis in Central Government Departments
The role of central departments is changing as they take on new responsibilities. This haspartly involved the setting up of new free-standing units such as the SEU and the Women’sUnit (WU) in the Cabinet Office. It is also a matter of a greater involvement by the centrein some key policy issues, such as the labour market.
This has already led to the recognition that more analysts and a wider range ofprofessional skills are required in central departments. For example, social researchers haveentered the Cabinet Office through the WU and SEU. Economists have entered theCabinet Office in larger numbers via the PIU. A greater presence of specialists in the centrecan help discharge traditional functions more effectively – e.g. identifying the need forand commissioning at an early stage “whole of Government” analysis, where appropriate.A wider range of specialist skills (for example in operational research and social researchas well as economics and statistics) may also be required in these central departmentsto identify opportunities for cross-cutting work in these specialisms.
Conclusion 8: the Cabinet Office and the Treasury should carry out a review of therole, numbers and types of specialists theyneed to support an enhanced “challenge”function to promote better planning ofanalysis by departments by end June 2000.
Better business planning withindepartments could improve the provision of analysis6.15 There is also a need to ensure effectiveprioritisation of analytical work withindepartments. Priorities for analysis should beset in parallel with departmental objectivesand PSA targets.
Box 6.3 Analytical BusinessPlanning in Departments
The responses to the project’squestionnaire suggest that mostdepartments have an annual planningprocess, but also have the flexibility tocarry out ad hoc research or analysisto much shorter deadlines.
Occasionally research and developmentof models is longer-term. For example,DfEE have published a focussed long-term research strategy. This wasdeveloped after direct discussions withall departmental Ministers and intendedto elicit the long-term informationrequirements of the department. Thedepartment has issued a prospectusintended to improve the way in whichit commissions, disseminates and usesresearch. The aim is to align academicresearch priorities more closely todepartmental needs by outliningdeveloping policy areas in whichresearch is needed.
6.16 It is important that departmentalbusiness planning fully integrates a strategyfor analysis and research. One of the strandsof the proposed peer review of departmentalbusiness planning processes (discussed inChapter 5) should be confirmation thatanalysis has been well integrated intomedium and long term business plans.
6.17 To achieve this, departments may needto set up cross-cutting units (or teams) totake an overview of policy areas (as theTreasury has done in relation to productivity)and which can ask difficult questions and callfor the use of analysis where appropriate.
6.18 There may be scope to link the report’sconclusions on business planning to the workthat departments already undertake in issuingdepartmental reports, which set out detailsof expenditure plans and policy initiatives.But detailed work on this area needs to awaitfurther development of the package of CivilService reform.
Conclusion 9: as part of the “better business planning” strand of the Civil Servicereform process, analytical strategies shouldbe developed by government departments.
Addi
ng It
Up
38
7.1 The conclusions in previous Chaptersshould help ensure that there is a vigorousdemand for analysis and that analysis is wellplanned in areas where demand may beweak. Even where analytical capability exists,it needs to be fully integrated with the policyprocess and to act as a driver of policydevelopment.
7.2 This Chapter considers what practicalsteps can be taken to promote better jointworking of various kinds – betweenprofessionals and policy makers and betweendifferent kinds of professionals.
7.3 The chapter is premised on theassumption – borne out by the interviewsand other evidence gathered for this study –
that successful analysis is a joint enterprise.Virtuous circles arise from successful jointworking. The provision of data and analysisleads to more policy questions being asked.This in turn leads to the demand for furtherdata and analysis.
7.4 There are two main dimensionsto more effective joint working:
• structural changes within departmentsto encourage closer working;
• improved training for policy makersand analysts.
7. INTEGRATING ANALYSIS WITH POLICY MAKING
Summary
Joint working between specialists and policy makers is central to ensuringthe integration of analysis with policy making.
There is no one right way of organising and managing specialists.Different models suit different types of policy area and departmentalcircumstances.
Policy makers and specialists need to talk a common language and eachgroup needs training to do this. The ability to commission and useappropriate analysis should be a condition of promotion for policy makers.
The different specialisms need to work more closely and effectively witheach other.
Integrating analysis
with policy
making
39
Policy makers and specialistsshould work more closely together7.5 The organisational structures withinwhich analysts ply their trade differ verysignificantly from one department to another.They range from full integration of specialistsinto the policy line – “bedding out” – to theirlocation in separate management units orseparate government departments. The trendin recent years has been towards increased“bedding out”. DTI, for example, went downthis route some five years ago (for economistsand statisticians). IR is at present activelyconsidering the merits of doing the same.Within the DfEE, labour market economistshave recently been bedded out under aSenior Civil Service (SCS) Grade 5 command,but other specialists continue to work in acentral unit.
7.6 The evidence gathered by this studystrongly suggests that there are some kindsof work that specialists do most effectivelywhen huddled together in a corner butothers that are best done sitting face to facewith policy makers and as part of a unifiedteam. The “huddled together” work is at themore technical end of the spectrum. Face toface structures help ensure that policy-makers“think like specialists” and make full use ofanalysis when developing policy. A frameworkfor deciding what works best is in Box 7.1.
Working in mixed teams7.7 The simplest model of joint workingis one in which there are no distinctionsbetween analyst and policy maker: the twohave become one.
7.8 HM Treasury comes closest to sucha model. In the Treasury, there are onlypostings – in each case filled by someonewho has the required blend of skills for aparticular set of responsibilities. Some posts
are filled by economists or other specialists,others by former generalists. The same postmay be filled in turn by generalist, economistand generalist – without changing incharacter. There remains a small core ofmicro-economists, looking principally atcross-cutting areas.
7.9 This structure is supported by a trainingprogramme, designed to ensure that allTreasury staff have the full range of skillsrequired to deliver excellent performance.Both generalists and specialists have had tolearn a more rounded set of skills. This hasrequired significant investment in trainingand large opportunity costs (each member ofstaff may need to give over several weeks ofthe year to training). No formal cost/benefitanalysis has been carried out of the impact.But the view of senior managers is that thenew regime has significantly improved thequality of analysis in submissions to Ministers.
7.10 The Treasury model serves thedepartment’s needs very well. But this modelmay not be readily applicable elsewhere,e.g. if the range of skills that must be broughtto bear on policy issues is wider. Differentdepartments require different specialist skills.
Specialists working alongside policy-makers7.11 A wide variety of arrangements forbedding out is found in differentdepartments. Within DETR, for example,economists working within a specialist SCSGrade 5 division form part of SCS Grade 3-led policy directorates. So too do statisticiansand social researchers in the DETR HousingDirectorate, facilitating close andmultidisciplinary joint working. In the airquality division, around 10 scientists arebedded out into a Grade 5 policy divisionunder a Grade 6 head, while environmentaleconomists are bedded into the policy lineunder a Grade 5 division.
Addi
ng It
Up
40
Integrating analysis
with policy
making
41
Box 7.1. Bedding Out – a Schema
This chapter describes the different organisational models for specialists operating inWhitehall today. The purpose of any particular structure must be to create the optimal flowof information and analysis between policy makers and specialists. For this to occur eachgroup should have a clear conception of the needs and capabilities of the other. In manycases this will mean integration of specialists into the policy management line at theappropriate level. One simplified way of thinking about which level is appropriate is shownin the diagram below:
The y axis shows the type of specialist advice on a range from technical (the application ofhighly specific techniques to problems) to mentoring (implying the general application ofreasoned argument and analysis to problems). The x axis shows policy areas on a rangefrom reactive (short term fire-fighting) to reflective (medium to long term work). Broadlyspeaking, the more day to day the pressures the more necessary it is to ensure thatspecialist advice is immediately available. The reverse is true for highly specialised advicedeveloped over a long period of time.
Any abstraction is necessarily a simplification and other dimensions will play a role. Thenumber of specialists in a department will be a key factor in determining whether beddingout is appropriate. And in some cases there are issues which are inherently ‘central’ innature e.g. appraisal and evaluation. Different models could be appropriate in differentpolicy areas within the same department.
More generally:
• departments will wish to retain some central capacity for specialists to provide specialistadvice in those areas where the level of demand would not be sufficient to justify adedicated resource;
• specialists would expect a variety of types of job within this schema during their careerin order to ensure that the full range of professional skills are maintained.
As technology advances, the debate over bedding out may become less central to theorganisation of specialists. For example, an increase in home working will tend to blur thedistinction between bedding out and centralisation.
Specialist and generalist merge
MENTORING
TECHNICAL
REACTIVE REFLECTIVE
Flexible teams to address technical policy problems
Bed out groups underhigher grade specialists
Bed out small groupsunder medium grade
specialist management
SPECIALIST
ROLE
POLICY TYPE
7.12 There is still a central core ofeconomists in the Chief Economist’sDirectorate. This is inevitably further removedfrom the policy process, but this teamservices policy areas which are too small tosustain their own dedicated economist andcan also take forward cross-cutting issueswhich would not otherwise be picked up.The involvement of specialists in the policymaking process is maximised under fullbedding out (e.g. with scientific input towork on air quality). But other arrangementsin DETR (e.g. in the Housing Directorate) alsolead to successful cross-disciplinary working.
Maintaining professionalism and integrity7.13 Other departments maintain a fullycentralised structure for specialists(economists, statisticians, social researchersand operational researchers), based aroundAnalytical Services Divisions or Directorates(e.g. DSS and Home Office). This promotesclose joint working between professionals,but leaves the link with policy colleagues inneed of good management.
7.14 More extreme is the position of GADand ONS: central units which are at adistance from the users of their services, bothin structural terms and physically. Professionaladvice in such cases is (slightly) harder toseek and (somewhat) easier to ignore. GADcharges departments for the actuarial advicethey commission, which gives departmentsan additional disincentive to seek actuarialadvice. But on the other side of the balancestand the manifest independence, perceivedintegrity, professional competence and espritde corps of these central units.
7.15 Physical propinquity also clearlymatters. The DSS Medical Policy Group, forexample, have been more closely involvedin policy making since moving into the samebuilding (and occupying the same floor) as
policy colleagues. Policy makers see medicalspecialists more as part of the team, less asstakeholders who need to be consulted.
There’s no single right answer on joint working7.16 It seems unlikely that there is any single“right answer” that will achieve the idealmeasure of professionalism alongside theideal arrangements for “joint working”. Theremay, however, be a spectrum of rightanswers that vary with the business needs ofdepartments. In general, the best balancemay be obtained through some combinationof a central specialist unit alongside a moreor less extensive “bedding out” of specialiststo policy teams.
7.17 Where central units exist, it is importantto ensure that analysts liaise closely withpolicy teams – that they are invited to teamplanning meetings, team awaydays etc.And that, where arrangements are lesslong-lasting, cross-cutting project teamsare formed in which analysts participateon a consultancy basis. In short, thatarrangements are in place withindepartments to secure the maximum benefitsof joint working and the maximum inputfrom specialist expertise.
7.18 Conversely, where analysts are beddedout, it is important that strong professionallinks are maintained between analysts andthe Head of Profession and that appropriatetraining and development programmes are inplace to ensure that professional identity andskills are maintained.
Addi
ng It
Up
42
Conclusion 10: departments should:
• review policy on bedding out once perParliament with a view to maintaining theright balance between central expertiseand bedded-out effectiveness;
• ensure that where analysts continue toform part of central specialist units, theyalso form an integral part of the work ofpolicy teams; and that where analysts arebedded out, appropriate professionaltraining and development programmesare in place.
Making policy makers better analysts7.19 Training for policy makers can helpjoint working by improving:
• their knowledge of specialist disciplines;
• the use of analysis in everyday work suchas submissions to Ministers;
• their understanding of what specialistshave to contribute to the policy process.
7.20 There are, of course, risks in piling anincreasing number of requirements ongeneralists. The list of requirements forpromotion to the Senior Civil Service isalready long and growing. There is,nonetheless, a case for adding to itcompetence in analysis. Part of the Treasury’srecent success in improving analysis is thatdemonstrated competence in analysis alongwith other key skills has become a conditionof promotion. Departments will of courseneed to determine in each case what form oftraining best suits their business needs: thoseworking on pensions policy, for example, willneed more of an understanding of actuarialissues (and less knowledge of accountancy)than counterparts in the Treasury.
Conclusion 11:
• other departments should put in placetraining arrangements for policymakerssimilar to those of the Treasury by end2001;
• demonstrated proficiency in analysisshould become a requirement ofpromotion to the SCS from April 2001.
Training specialists to be better policy makers7.21 There is also a case for trainingspecialists more in the policy process toenable them to become more skilled inselling their wares to policy makers and moreknowledgeable about the policy process.Specialists who make themselves known andadvertise what they can do for policycolleagues are more likely to be valued.
Conclusion 12: new forms of training for specialists should be developed to supportjoint working (e.g. courses on “partnershipworking”) and introduced by the CMPS fromend 2000.
Specialists need to work together – and often do7.22 Many departments brigade all theirspecialists together in Analytical ServicesDivisions. This gives strength in numbersand the potential to challenge aspects of thepolicy process from a combined analyticalperspective. It also promotes closer workingbetween specialists. Even where specialistsare not brigaded together, they can alsoco-operate closely. For example, theDepartment of Health (DH) model for NHSwaiting lists discussed in case study 8 ismaintained jointly by economists, statisticiansand operational researchers. Sometimes theChief Economist in a department will alsoserve as the Chief Statistician.
Integrating analysis
with policy
making
43
7.23 There is some overlap of professionalskills. Work that is done by an economist inone department may equally well be carriedout by a statistician in another. AcrossWhitehall, many different kinds of specialistmanage external research contracts. This isalso the case in the outside world. Forexample, at the IFS it is economists ratherthan statisticians who tend to perform datarelated tasks such as ‘cleaning’ andmanipulating data sets. It clearly makes sensein a large department for individuals tospecialise in different roles. But this overlap ofspecialist skills reinforces the desirability ofclose joint working within departments.
Leaders of specialists must work more closely together7.24 The arrangements for joined-upworking between specialists in departmentsare not paralleled at an institutional level orbetween Heads of Profession.
7.25 Heads of Profession (or, where these donot exist, Chairs of Management Groups) arein a unique position both to overseedevelopments across their own professionaldomain and to promote cross-cuttingworking between professionals.
7.26 Periodic joint conferences and jointmeetings between Heads of Profession, whonever routinely congregate at present, wouldgive a further mechanism for taking anoverview of analytical work within Whitehall.Such a forum could act as a furthermechanism to help ensure that the necessaryanalysis to support policy development isundertaken. Where gaps were identified, thesemight become the subject of bids to theseed-corn fund recommended in Chapter 5.
7.27 The Heads of Profession most closelyaffected by the recommendations of thepresent report are the Heads of GES, GSS,GORS, GSR and the Government Actuary.But other Heads of Profession inGovernment, such as the Chief Scientist orChief Medical Officer, might also have aninterest and wish to attend.
Conclusion 13: there should be periodic joint meetings of Heads of Profession. Headsof Profession should consider the case foroccasional joint conferences betweenspecialists groups (i.e. GSS, GORS, GES, GSR).
Addi
ng It
Up
44
8.1 Chapter 7 discussed ways of ensuringthat analysts work closely with policy makers.A related requirement is that there is a supplyof good quality analysts available toGovernment. The purpose of this chapter isto discuss how problems in the recruitmentand retention of good analysts impact onanalysis and modelling and what might bedone about them.
It’s increasingly difficult to attract and retain specialists8.2 Most departments currently haveexcess demand for economists. Although,historically, there has always been a shortfallin the supply of economists, the position hasdeteriorated in recent years. Much of this isdue to the increased demand for micro-economists (and particularly those withknowledge of the regulated industries) fromprivate sector economic consultancies.
8. BRINGING IN AND BRINGING ON TALENTEDSPECIALISTS
Summary
There are particular difficulties at present in the recruitment and retentionof economists (and to a lesser degree other specialists). The Governmenthas already responded to this by raising the entry level rates of pay – andthe internal market within the Civil Service may bid up rates of pay further(as may external competitors for good quality entrants). Rates of pay are,however, already attractive to academics.
There is scope to amend the entry requirements for economists, so thatnot all entrants need be potential members of the Senior Civil Service.There is also scope to offer more personal promotions to those withspecialist expertise.
Departments need to sell more strongly the attractions of the publicservice as a career – the sense of making a valued contribution toimportant questions of public policy as well as the flexibility in workingpatterns offered by the Civil Service (by comparison with, say,consultancy). There is scope also to offer more varied careersto specialists, offering academic secondments and specialist careersin technical modelling work.
Bringing in
and bringing
on talented
specialists
45
8.3 Results from the 1999 Civil ServiceSelection Board (CSSB) process for recruitinggraduate economists illustrate the scale of theproblem. Despite initial bids fromdepartments for 120 candidates in February(and 60 bids in July) only 54 appointments(and 8 in July) were made. The recruitmentproblem is generally common acrossdepartments, although in some (such asDFID) the non-financial rewards make itrelatively easier to fill posts – many entrantshave a strong commitment to the DFIDpolicy agenda.
8.4 As with economists, there has also beena significant shortfall between the number ofbids for Assistant Statisticians and thenumbers recruited through CSSBcompetitions. In 1999 the total number ofbids was 122 and the number of offers madeto applicants was 85. (In 1998 57 AssistantStatisticians were appointed through theannual competition and there were 72 bids.)
8.5 The demand for social researchers hasincreased across Whitehall. For example, boththe CO and the Home Office have recentlyincreased their social research capacity.Recruitment is slightly easier than foreconomists and statisticians. It is carried outdirectly by departments. For example, DfEEhad 160 applicants for 15 social researchposts in their most recent recruitment roundand filled them all. Retention withindepartments is a bigger problem as Whitehalldepartments compete on pay to attract thebest people.
8.6 Recruitment of operational researchersis also at present very problematic. 164professional OR staff are currently in post indepartments other than MoD. Those samedepartments declared 30 vacancies in theApril/May recruitment round, of which 21were filled, but in August they again declaredalmost the same number of vacancies (26)and filled only 11.
8.7 Actuaries are centrally recruited byGAD. This, together with GAD’s “hardcharging” regime – which involves chargingdepartments for staff costs incurred whenadvice is given – gives greater flexibility onpay. New recruits are offered starting salariesin line with market rates. But those in moresenior posts are offered considerably lowersalaries than their private sector counterparts.There are accordingly some difficulties ofretention. As with other specialists, however,the attractions of government service (e.g. thesheer variety of subject matter) outweigh thefinancial prospects of the private sector forsome high quality performers.
We should be more flexible in recruiting graduate economists8.8 The main method of graduate “faststream” recruitment to the Civil Service fornearly all economists and many statisticiansis through CSSB. The criteria used in theselection process focus on all-round skills(e.g. management, technical and inter-personal skills) to evaluate whether eachcandidate has the potential to reach theSenior Civil Service (Grade 5 and above).
8.9 The present criteria for entry undulynarrows the potential stock of successfulapplicants. Some analysts may have particularskills such as a detailed knowledge of cutting-edge techniques within their chosen field ofexpertise. A diverse range of skills is neededin most organisations. For example, there isno evidence that private sector companiesonly recruit people who are managementmaterial. Indeed, the GSS no longer recruitsgraduate new entrants solely to “fast stream”posts. The GES should follow this example.
Conclusion 14: GES should recruit some people who have a strong aptitude forquantitative methods suitable for some Grade7 posts but who do not necessarily have theall-round potential to reach Grade 5.
Addi
ng It
Up
46
Departments should makemore use of direct entrants at all levels8.10 The overall scale of direct recruitmentof Grade 7 (middle manager) economists hadnot changed significantly over the past 20years at the time of the last review of GES in1994. Between 1978 and 1994 the numberof economists recruited directly into the CivilService at this grade has been around halfthose internally promoted (see Figure 1).Direct recruitment of economists to theSenior Civil Service has been and remainsvery rare.
8.11 Government service can offer a rangeof non-financial rewards to potential Grade 7direct entrants. The GES offers opportunitiesto contribute to big issues in a way thatmatters and a more flexible balance betweenworking and home life than is offered by, say,consultancies or the City. Theseconsiderations can outweigh salarydifferentials for some people. And in the caseof academics, civil service rates of pay maythemselves be an attraction. The Treasuryand ONS have recently recruited economists
successfully at Grade 7 level from academiaand the private sector.
8.12 Increased recruitment of economistsat Grade 7 level is not a panacea. The entryconditions for Grade 7 level tend to be lessrigid than those of CSSB (a 45 minuteinterview rather than 2 day’s worth of testsat an assessment centre). This results in ahigher risk of recruiting poorer performers.And good induction programmes arerequired to ensure that direct entrants canbecome effective quickly. But direct entry atGrade 7 level is a valuable supplement to theentry route through CSSB which departmentsshould explore further. Indeed the GES is atpresent making a concerted effort to attractexternal applicants at all levels (6).
8.13 Direct entry at Grade 7 is morecommon for social researchers. For socialresearchers the pay differential between theGovernment and the private sector is lesssignificant (at least for less experiencedpeople) and it is possible to attract peoplefrom academia and, to a lesser extent,market research companies. There is evidenceof recent successful recruitment to Whitehall
Bringing in
and bringing
on talented
specialists
47
0
5
10
15
20
25
30
35
40
1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998
Recruited PromotedSource: EGMU
Figure 1: GES Promotions and Direct Entrants to Grade 7 Posts, 1978 – 99
from outside Government. For example, thehead of research at the Home Office is a newrecruit to the Senior Civil Service fromacademia. There are also examples ofsuccessful direct recruitment within theoperational research community. DH, forexample, has recruited people from industry.Actuaries are recruited by GAD from theprivate sector at Grade 7 level on a fairlyregular basis.
Conclusion 15: departments should recruit more external applicants directly intospecialist (and particularly economist) posts,both at Grade 7 level and above.
Restrictive nationality rules are outdated8.14 The available pool of labour is crucial indetermining how many good quality analystscan be recruited into the Civil Service.However, regulations on nationality canprevent non-UK nationals entering the CivilService, resulting in a much smaller pool oflabour on offer. This is particularly importantgiven the decline in the number of Britisheconomics students enrolling on PhD coursesin UK universities. The top economicdepartments are averaging less than two newBritish PhD students per year (7).
8.15 These nationality restrictions areoutdated. Indeed the Cabinet Office iscurrently attempting to change them.However, to do this will require legislationat a time when the Parliamentary timetableis tight. There is, though, nothing toprevent departments from employing moreCommonwealth and European Communitycitizens without contravening currentrequirements for non-reserve3 posts. DFID,the Treasury and ONS have started to do so.
Conclusion 16: departments should seek to broaden the pool from which specialistsare recruited to include foreign nationalspecialists wherever possible.
More varied career structureswould help to retain good specialists8.16 Promotion prospects for specialists arelimited. Many economist Grade 7s leave theGES around one year after promotion. Lackof promotion prospects to the Senior CivilService is one of the main reasons cited forleaving. On average 5 people a year havebeen promoted to SCS Grade 5 posts since1978 (see Figure 2). For the period 1978-1994, the median age of those recruited was41. (More recent statistics are not available onage of promotion.)
8.17 This pattern of promotion can send outundesirable signals to those currently withinthe GES. It may create the perception thatpromotion within the GES is based on lengthof tenure rather than merit. Above Grade 5there are even fewer opportunities forpromotion mainly because of the removal ofnumerous SCS Grade 3 and Grade 2economics posts during the 1980s. Currently,there are 10 GES members at Grade 3 andabove. This compares with 17 posts in 1992.
8.18 For statisticians, an indication ofpromotion prospects within the GSS is givenby Figure 3, although the graphs do notreflect internal promotions withindepartments. The figures reveal that thepotential opportunities for promotion to theSCS at Grade 5 level are limited.
8.19 Within GORS, promotion opportunitiesto Grade 5 are even more limited than in theGES or GSS, as the Head of Profession rolehas been downgraded or has disappeared inseveral departments. Any OR specialist who
Addi
ng It
Up
48
3 Reserve posts are those which only British citizens can apply for because of security and other reasons.
wishes to progress beyond Grade 5 mustchange career direction and work as anadministrator. Similarly, the small number ofsenior posts within the GSR (a Head ofProfession post at Grade 5 level in mostdepartments with a strong interest in socialresearch) encourages some experiencedresearchers to leave the Civil Service.
8.20 There is a case for making more use ofpersonal promotion for specialists. This wouldgive them a greater incentive to remainwithin the Civil Service and in specialistgrades if that is where their talents are bestdeployed. Long-serving analysts can act asthe collective memory of a department andbuild up unrivalled expertise in a particulararea. There are already some examples ofpersonal promotion among professionals inthe Civil Service (e.g. in the Home Office).Civil Service reform should enable andencourage other departments to followthis path.
Conclusion 17: departments should introduce more personal promotion to allowindividuals to choose between specialist andgeneralist career progression paths.
8.21 More varied career paths need to beoffered to specialists to show that theircontribution is valued.
8.22 The number of secondments and otherlinks to academia could be increased.Secondments to academia help to updatespecialists’ skills and should be part of normalprofessional development in posts wheretechnical skills are important. Links withacademia also heighten a sense ofprofessional identity – an important factorfor many specialists.
Box 8.1. Ways of Keeping UpTo Date
Departments have found a range ofways to help keep staff up to date withacademic thinking. DSS run a regularlunchtime seminar series for analysts andpolicy makers jointly with John Hills atLSE. And many departments runacademic panels which meet regularlyto discuss papers on research andmodelling. Customs and Excise believethat, because the number of academicsworking in the field and the range oftopics for discussion has proved too smallto run an effective panel, one wayforward might be to establish a jointHMT/IR/CE forum focusing on tax issues.There may similarly be scope for joined-up working in other academic panels.
Bringing in
and bringing
on talented
specialists
49
0
2
4
6
8
10
12
1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998
Source: EGMU
Figure 2: Promotion to Grade 5 Posts, 1978 – 99
Addi
ng It
Up
50
0
5
10
15
20
25
30
1992 1993 1994 1995 1996 1997 1998 1999
Transfer
Promotion
Secondment
Inter-departmental moves for Grade 7s
0
1
2
3
4
1992 1993 1994 1995 1996 1997 1998 1999
Transfer
Promotion
Secondment
Inter-departmental moves for Grade 6s
0
1
2
3
4
5
6
7
1992 1993 1994 1995 1996 1997 1998 1999
Transfer
Promotion
Secondment
Inter-departmental moves for Grade 5 and above
Source: GSS
Figure 3: Inter-departmental moves for statisticians
Bringing in
and bringing
on talented
specialists
51
8.23 Chapter 11 discusses links with theoutside world in more detail. But within thespecific context of encouraging retention,it may also be worth encouraging:
• analysts to publish their work in academicjournals/speak at conferences;
• more collaborative work with academics.
Box 8.3. FCO Secondment to BP
In 1997 the FCO seconded a Grade 7economist to the oil company BP.The benefits were:
• for BP, the secondment not onlyprovided an insight into the Whitehallmachinery but also an extra well-qualified employee at less than themarket wage;
• for the FCO, it provided anopportunity for one of its employeesto increase his skill set at no cost tothe department;
• for the individual it was an opportunityto broaden his experience and develophis career further whilst remaining inthe public sector.
Conclusion 18: departments should promote more vigorously secondments toand from the private sector and academia;and should continue to promote other linkswith academia.
8.24 Secondments to and from the privatesector can bring benefits to all parties.
8.25 Skills in modelling and quantitativetechniques are transferable betweendepartments. There is scope for individualanalysts to improve the diversity of careersthrough movement between departments.For statisticians (see Figure 3) andeconomists, the movement betweendepartments is already considerable.This should be maintained.
Higher pay may need to be offered to good analysts8.26 For all the non-financial rewards ofgovernment service, it may ultimately provenecessary to offer higher pay to recruit andretain good analysts.
8.27 Responsibility for pay is delegated toindividual departments. This has resulted in arange of different pay incentives being usedacross Whitehall to improve recruitment andretention. Some specific examples include:
Box 8.2. Increasing Human Capital and a Sense of ProfessionalIdentity
There are examples of departmental links with academia which not only lead to an increasein specialists’ human capital, but also heighten specialists’ professional identity:
• sponsorship of specialists to undertake second degrees is widespread, although normallyMScs (the case for sponsoring PhDs is less strong);
• economists in the Home Office and the former DoT have been encouraged to publishresearch in a personal capacity, and to speak to academics about their work;
• DH is seconding an analyst part time to academia to develop its long-term care model;
• the Women’s Unit in the Cabinet Office employs two academics on a part time basisto oversee their research on gender issues.
Addi
ng It
Up
52
• higher pay for Assistant Economists withinC&E;
• a recruitment and retention allowance ofaround 20-25% to junior staff in DSS;
• individual additional allowances in DfEEto analysts who have been offered higherpaid posts in other departments;
• wide variation in pay for social researchers.The pay scales for Research Officers havescale minima and maxima in NationalSavings that are both around 25% higherthan in DfEE.
8.28 Departments face a number ofproblems in using pay to tackle difficultiesin recruiting and retaining staff:
• departments’ current running cost budgetsare unlikely to be able to bear a significantincrease in specialists’ pay (unless thenumber of specialist posts is reduced toensure that the total salary bill remains thesame). A Society for Business Economists’Survey suggested that in 1999 the mediansalary difference between the wholesample and Government was around£8,500. To come close to matching thisdifferential would place significant burdenson departments;
• marginal changes in pay might beineffective. There is a risk that the privatesector would just increase pay to restoredifferentials.
8.29 In some specialist areas, however,such as regulatory economics, difficulties ofrecruitment within Government are now verysevere. If the Government wishes to employeconomists directly (and this is likely to becheaper than contracting advice out) theremay ultimately be no alternative to increasingpay for specialists to near market levels.
8.30 Such a development would haveknock-on consequences. It might discourageanalysts in future from applying for generalistposts, if these became significantly less wellpaid. The scheme might thus only work ifthere were opportunities for generalists toearn similar salaries – perhaps if theydemonstrated that they were analyticallycompetent. The implications of such a step(and the alternative of accepting adiminished exchange between specialist andgeneralist postings) would all need carefulconsideration. The first step, however, mustbe to consider further the ways of improvingthe recruitment and retention of specialistswhere the Government faces a severeshortage.
Conclusion 19: professional groups should immediately review the impact of devolvedpay structures on retention and inter-departmental moves and consider whether aGovernment wide increase in pay is requiredto ensure a satisfactory supply of high qualityanalysts.
Box 8.4. Frontier Economics Secondment to the Cabinet Office
Earlier this year, Frontier Economics, an economic consultancy, seconded a member of staffto the PIU in the Cabinet Office for a period of three months. The benefits were:
• for the individual, an opportunity to become acquainted with the workings ofGovernment and to obtain considerable knowledge across a number of policy areas;
• for the Cabinet Office, a way of accessing technical expertise over a continuous periodof time and less costly way of buying in consultancy time;
• for Frontier Economics, an invaluable set of contacts in an industry where reputationis all-important.
9.1 Previous Chapters have set out thefoundations for ensuring good analysis inWhitehall: a vigorous demand supplementedby improved business planning; high qualityanalysts who work alongside policy makersand help drive the policy process; and whoare recruited across specialisms in sufficientnumbers.
9.2 The present Chapter addressesproblems specific to data collection,access and use.
9. HOW TO GET MORE AND BETTER DATA
Summary
Soundly based analysis and modelling requires access to and use of gooddata. The uses of data range from simple descriptive analysis to detailedmodelling based on complex longitudinal data sets.
Relatively simple improvements in the access to and use of data could paybig dividends: e.g. greater use of basic descriptive statistics, more use ofadministrative data, greater matching of different data sets; and more useof imputation within data sets. In many of these areas, the ONS is alreadydoing valuable work.
There is scope for data in key policy areas, such as poverty and socialexclusion, to be released more quickly, and for greater use to be madeof panel and birth cohort studies. More use should be made of pilotsin policy development to test what works before options are rolledout nationally.
The White Paper Building Trust in Statistics (4) sets out the Government’splans to establish improved arrangements for the accountability,governance and quality of official statistics and the future role of ONS.
How to
get more
and better
data
53
Better use could easily be made of existing data9.3 Departments should be encouragedto make more use of basic data descriptionin the policy formulation process. Themechanism for this should be disseminationof good practice by the CMPS.
9.4 Powerful insights can be obtained fromsimple tabulation or graphical representationof key data. Simple analysis of survey data canreveal, for example, the pattern of spendingon energy across different household types,or levels of car ownership. Sometimes thedata required to set the context for policydevelopment may need to be assembled froma variety of different data sets.
Addi
ng It
Up
54
Box 9.1. Bringing Data Together
We All Need Pensions
The Pension Provision Group (PPG) report We all need pensions (8) was published in April1998, around 6 months before the Government’s Pensions Green Paper (9). It covered awide range of basic issues in pension provision, ranging from “incomes in old age” to“the labour market context” to “risk and efficiency in pension provision”.
This work helped set the context for subsequent policy development. None of theinformation was new to policy makers but the publication provided chapter and verse forthe stylised mental map of the pensions world with which policy makers in DSS need toequip themselves. It represented a very helpful assembly of different data sources in asingle place. The existence of the PPG also caused DSS to give higher priority to basic datacollection than otherwise would have been the case (because it would have been crowdedout by the pressure of more urgent work).
Drawing Together the Big Picture on Smuggling
A range of methods has been used to measure the overall extent of smuggling of tobaccoand alcohol:
• the drinks industry has conducted (and continues to conduct) quayside surveys ofloaded vans in Calais to give a broad indication of the volumes of alcohol being broughtinto the UK from the EU. This focuses on cross Channel smuggling of alcohol;
• Customs officers conduct surveys measuring the extent of smuggling by stoppinga random sample of vehicles returning to the UK from the Continent;
• the International Passenger Survey (run by the ONS) provides time series survey dataon movement of passengers in and out of the UK and data on cross-border shopping;
• C&E run an Air Passenger Survey to pick up smuggling by air. Customs officersdetermine through covert surveillance high risk luggage on particular flights. This is thenexamined on a 100% basis for contraband;
• although freight is a growth area for cigarette smuggling, it is impossible (for practicalreasons) to do a sample survey. Seizures are intelligence-led and the success rate isincreasing, which in itself provides useful information. (Around 0.1% of freight trafficis estimated to carry contraband.)
9.5 There also needs to be better accessto large data sets across departments.A wide range of survey and other datasources is already used by departments.However, both the availability and useof official Government surveys acrossdepartments are not as widespread as theycould be. Improvements in computingtechnology will make it increasinglystraightforward to store and access data from,for example, the Family Resources Survey(FRS) or the General Household Survey(GHS). Providers of data should design,document and structure them to facilitate awide variety of possible uses, and not simplyto meet the immediate policy demand.
9.6 Some investment may be required toincrease specialists’ familiarity withmanipulating large data sets. And ONS willneed to make its own surveys, such as theFamily Expenditure Survey (FES), as accessibleas possible to attract new users unfamiliarwith the database (e.g. in terms ofsupporting documentation). Departmentalstatisticians might also play an advisory role ifother specialists lack the skills required.
Conclusion 20: departments should review, by June 2000, whether they need thecapability to access and use largeGovernment surveys such as the FRS and,if so, put in place the necessary capacityby June 2001.
9.7 More effective use could be made ofadministrative data. Government collectsvast amounts of administrative data, rangingfrom information on social security claims todata on the work of the courts and claims forlegal aid. Whilst administrative data of thiskind feed into specific models, little use ismade of such data beyond the departmentswhich collect it. (The ONS JUVOS4 data seton the claimant count appears to be anexception to this but is difficult to obtain and
complex for non-ONS analysts to use). This isin stark contrast to practice in North Americawhere use of administrative data iswidespread.
9.8 An important barrier to wider use ofadministrative data is confidentiality. Thisis already being addressed:
• the SEU Policy Action Team 18 (set upto devise a strategy for small area data)met with the Data Protection Registrar toexplore whether it would be possible tomake more use of administrative data at asmall area level. It was concluded that theData Protection Act is not of itself a barrierto information sharing, but only requiresdata controllers to think much morecarefully about how they collect anduse data;
• the GSS Committee on Social Statisticsis exploring how the potential foradministrative data might be unlocked.In particular, efforts are being made toidentify what use of administrative datacan legitimately be made within thepresent legal framework.
9.9 An alternative approach would involvethe more widespread sharing of secondarysources based on administrative data. This isan approach that has been adopted in Japanwhere there are also strict confidentialityregulations applying to the analysis ofprimary sources of administrative data. As aresult, the Japanese labour ministry is able tomake detailed employment statistical seriesbased on administrative data available forsecondary analysis.
Conclusion 21: ONS and SEU shoulddisseminate more widely their work onadministrative data to specialists, for exampleat the GES conference or through theproposed Modelling Network (see Chapter10), by the end of 2000.
How to
get more
and better
data
55
4 The JUVOS (Joint Unemployment and Vacancies Operating System) cohort is a five per cent sample of computerised claims for
unemployment-related benefits going back to 1983.
Conclusion 22: departments collecting administrative data should consider releasingsecondary sources based on it.
Different data sources couldbe used more creatively to derive the big picture9.10 Data sets could be used more creativelyby matching different sources.
9.11 In addition to pooling survey andadministrative data, there is potential forlinking information from surveys ofbusinesses, individuals and households.
9.12 A GSS task force is already tackling theissues involved in matching data sets. It islooking at issues such as common identifiersacross data sets at an individual and regionallevel and the barriers to linking data sets.Outside Government, a team at theDepartment of Applied Economics atCambridge University is currentlyinvestigating the potential for matchingthe FES with the FRS.
Conclusion 23: ONS should promulgatewidely the recommendations of the GSS taskforce on matching data sets to all specialistsinvolved in modelling work and data analysis.ONS should do this through workshops andseminars to be held before the end of 2000and periodically thereafter.
9.13 Gaps in data sets can be filled byimputation5. This technique was the subjectof a GSS Task Force which produced a reportin 1996 (10). It reviewed the range ofmethods that have been proposed and gaveexamples of their use. Nonetheless,imputation remains little used in Whitehall.This suggests that there may be scope todisseminate the Task Force’s conclusionsmore thoroughly and to a wider audience.
9.14 One example of how imputation mighthave been used is suggested by the casestudy of long term care of the elderly (casestudy 7). Instead of using Council Tax bandsas a proxy for property values, it would havebeen possible to impute property valueswithin the FRS more explicitly by drawing onother information included in the survey suchas the region, and number of rooms.
Conclusion 24: ONS should encourage specialists to be more creative in their useof data to help fill specific data gaps withinsurveys. Through seminars and workshopsexamples of best practice should be exposedto a wider audience.
Data should be released more rapidly9.15 The academic community can onlyhelp with the analysis of data sets after theyare released. In comparison with the releaseof data on the macro economy, there areoften significant time lags between microdata being collected and its availability foranalysis. This means, for example, that therewill be no external analysis of changes untilaround 2001 in low incomes following themajor policy changes being made now tohelp alleviate poverty.
9.16 One way of minimising the time lagsmight be through the more frequent releaseof household and other survey data, wheresample sizes permit and demand is present.Another option would be to continue torelease data on an annual basis but to makeit available more quickly.
Conclusion 25: departments responsible for surveys covering income should carry outa review by end June 2000 of the costs andbenefits of more frequent and timely datarelease.
Addi
ng It
Up
56
5 Imputation involves estimating missing variables or observations within a data set, using variables for which there is information
to help predict the missing data.
Testing what works9.17 More use should be made of panel dataand birth cohort studies. Cross-sectional datacan provide descriptive statistics at a point intime, but more powerful conclusions can bedrawn from longitudinal surveys whereinformation from survey respondents iscollected over a period of years (see Box 9.2).
9.18 Longitudinal data can be used toanalyse the potential impacts of policies overtime. They permit analysis of how policyinterventions may affect future outcomes.Birth cohort studies are useful whendetermining the impact of early interventionson key transitions in later life.
9.19 The UK is fortunate in having one ofthe most long-established birth cohort
studies – the National Child DevelopmentStudy (NCDS) – as well as a number of goodpanel data sources like the British HouseholdPanel Study (BHPS).
9.20 ONS and other departments areworking on a strategy for longitudinal data.This will both bring together Government’sneeds for longitudinal data about people,families and households and identify prioritiesfor funding across departments.
9.21 However, to draw successfully on suchsurveys, policy makers and specialists needto be aware of the available data sourcesand helped with their use. This is not only amatter for statisticians. All analysts should beaware of these studies in order to influencethe future design of such data sources onan ongoing basis.
How to
get more
and better
data
57
Box 9.2. IFS Analysis Of Pensions
The IFS report Partnership in Pensions: An Assessment (11) provided a commentary onthe Government’s Green Paper on pension reform. The IFS analysis used longitudinal datafrom the British Household Panel Survey to estimate the size of the market for StakeholderPensions. Its analysis showed that:
• the majority of full-time employees earning between £9,000 and £18,500 (the targetgroup for Stakeholder Pensions) on a regular basis already have a private pension.Many of these might be ill advised to switch to a Stakeholder Pension if they havepaid up-front charges for their current pension;
• people earning between £9,000 and £18,500 who do not have a private pension areless likely to have stable employment and earnings than those who do. People whoseearnings fluctuate are likely to be better off with the State Second Pension (SSP) thanwith a private pension unless the Government continues to make a contribution totheir pension fund as an alternative to credits for a SSP;
• people who earn between £9,000 and £18,500 and do not have a private pension areless likely to have any other form of savings. The IFS concluded that, given they mayhave less stable employment patterns, these people may prefer to hold any savings ina more liquid form than a pension.
The Government’s view was that the dynamic aspect, whilst important, was not criticalto the success of the policy. The aim of stakeholder pensions is to put people in a positionwhere they can save when they can afford to do so, while ensuring that they are properlyprotected if they have to stop paying in (see case study 5 for a discussion of the DSS analysis).
Conclusion 26: ONS and lead departments should promulgate their strategy forlongitudinal data to analysts acrossGovernment through a series of workshopsand seminars.
Conclusion 27: GSS should disseminate best practice in the use of longitudinal data.
9.22 Pilots can play an important role inpolicy formulation and help evaluate policiesbefore national roll-out. Pilots can workthrough:
• random assignment, which involvesplacing individuals at random in a controlgroup and deliberately sheltering themfrom the policy change. After theintervention they can be compared withthe ‘treatment’ group who receive thepolicy, much like a clinical trial;
• geographically limited trials, which involvechanging policy in limited geographicalareas. This enables the outcomes in thoseareas to be compared with those inmatched “control” areas where policy didnot alter.
Conclusion 28: CMPS should set up a working group, by June 2000, to exchangeexperiences between departments andexplore the future role of pilots. Best practiceguidance should set out the expectation thatdepartments should use pilots, as part of thepolicy appraisal process, wherever appropriate.
Identifying and filling the data gaps9.23 There are few areas where there is acomplete lack of data. But there are somesubjects on which it would be desirable to havemore information. The main gaps relate to:
• company accounts data;
• information on individual assets andwealth (particularly for the elderly);
• small area data.
9.24 Of course data gaps emerge over time,and it is likely that more will emerge aspolicies change and new issues rise up thepolicy agenda. Data gaps emerge as socialcircumstances change. For example,registration of marriages is no longer sufficientto build up a good picture of the process offamily formation. It is therefore importantthat there are ongoing structures to ensurethat data gaps are picked up as they arise.
Addi
ng It
Up
58
Box 9.3. Use of Pilots
There was a random assignment element within the DSS pilots for the New Deal for LoneParents. Within the geographical areas where the pilots were run, only claimants with lowNational Insurance Numbers (NINOs) were invited to interview and to participate in theprogramme during the pilot period (leaving those with high NINOs as a control group).
A geographically limited pilot has been adopted for Educational Maintenance Allowances.The aim is to test whether financial incentives will encourage more young people fromlow-income families to stay in education beyond 16. Young people living in the pilot areaswill be eligible, depending on parental taxable income, to receive an EducationalMaintenance Allowance (EMA) if they attend full-time courses at school or college. Anotherexample, the Earnings Top Up (which extended Family Credit to those without children),was trialled over several years to assess its impact on local wage-setting by employers.
9.25 There are several ongoing initiativeswhich should help address these gaps:
• ONS runs a formal “gaps” exercise tounderpin its own work programme. A one-off CSO exercise conducted around sixyears ago led to collaboration between theEconomic and Social Research Council anda number of departments on time-usestatistics (covering leisure time activitiesand the role of non-traded activities in thehousehold). The exercise has since beenrepeated by various GSS committees.Some, such as the GSS committee onRegional Statistics, have formally reviewedthe earlier findings every two years;
• the SEU PAT 18 group on Better Informationis co-ordinating an exercise to bring togethersmall area data. The objective of the groupis to prepare a coherent cross-Governmentstrategy to collect, on a consistent basis,more up to date information on deprivedareas. As part of its work the group hasdrawn up a list of small area socialexclusion indicators and the data sourcesthat might be used in the compilation ofthe indicators. These indicators (around60-70 in total) cover all dimensions ofsocial exclusion, from economicdeprivation to the physical environment.
Conclusion 29: ONS should carry out a review, by June 2000, to establish whetherthey should set up a team to co-ordinate thecompilation of local area data outside thesocial exclusion arena.
Conclusion 30: ONS should bring together interested departments, by June2000, to assess whether there is a businesscase for a regular survey of individual wealthand assets.
Conclusion 31: the National Statistician should ensure that regular review of gapsforms an integral part of the new NationalStatistics planning process, from April 2000.
The role of the centre – co-ordination, oversight and ownership9.26 In the context of data collection andanalysis, the ‘centre’ includes the ONS aswell as the Treasury, the Cabinet Officeand No. 10. Its role is essentially one of:
• co-ordination;
• oversight of cross-cutting issues;
• encouragement of departments to worktogether where appropriate;
• ensuring that statistical information is seenas a national asset and collected, managedand disseminated as such.
9.27 ONS has a particularly crucial roleto play:
• providing advice about the availabilityof data sets;
• acting as a central holder of Governmentdata;
• providing data quality assurance;
• setting priorities for future data collection.
9.28 There is a mix of ownership of surveysacross departments and the centre. In somecases, ONS is responsible for holding anddeveloping official surveys (e.g. the FES).In other instances, surveys originating fromone department’s needs are run by thatdepartment (e.g. DETR runs the Surveyof English Housing).
9.29 An early task for the NationalStatistician, in conjunction with departmentalHeads of Profession, will be to review theextent to which the principles on location ofresponsibilities set out in Building Trust inStatistics (see Box 9.4) are reflected in thecurrent distribution of ownership of datasets across Whitehall.
How to
get more
and better
data
59
9.30 Problems can arise when the data tobe collected spans a number of departments.There are a number of ways of tackling this.Two examples are set out in Box 9.5.
Conclusion 32: as part of the new National Statistics planning process, ONS/GSSshould review, by June 2000, whether morecross-departmental data collection shouldbe undertaken along the model of the SEUand IGCB.
Addi
ng It
Up
60
Box 9.4. Building Trust in Statistics
The September 1999 White Paper Building Trust in Statistics (4) set out arrangements fordeveloping public confidence in official statistics. In particular it outlined the accountabilityand governance arrangements which the Government intends to introduce. A StatisticsCommission, which will be independent of both Ministers and the producers of NationalStatistics, will play a key role in quality assurance and priority setting, ensuring that officialstatistics are trustworthy and responsive to public needs.
The White Paper advocates that the distribution of National Statistics activities acrossdepartments should be determined on a case by case basis, weighing up the advantagesand disadvantages of their being centralised in the ONS or left in other Governmentdepartments or other public bodies. It proposes that any assessments should take accountof the breadth of the customer and the supplier base, the knowledge base and synergieswith other activities.
Box 9.5. Models for Joint Working to Deliver Cross-cutting Data
Social Exclusion Unit Policy Action Team on Better Information (PAT 18)
PAT 18 is made up of a group of experts on local area data from Whitehall, academiaand the private sector. It has been asked to report on:
• the scope for a coherent cross-Government strategy to get more up to date andconsistent information on deprived areas;
• how this could be done without generating undue bureaucracy;
• evidence on good practice by individual local authorities and how this could be spreadmore widely;
• the role of Regional Development Agencies in aggregating area information.
Their draft report sets out a vision for encouraging the use of information in analysingoptions for neighbourhood renewal.
Interdepartmental Group on Costs and Benefits
As the Government’s Air Quality Strategy crosses a number of departmental boundaries,an interdepartmental approach to analysis and policy formulation is crucial. Whilst theDETR is the lead department, it relies on analysis from other departments. For example,DTI provide forecasts of future energy demand and DH feed data in on health.
10. IMPROVING THE QUALITY OF MICROECONOMICMODELLING IN GOVERNMENT
Summary
The key to the successful use of models is to understand their limitations.They can only be as good as the data on which they are based. They mustnot be impenetrable “black boxes” to analysts or to policy makers. Theoutputs of models need to be seen as information which helps improveunderstanding of e.g. the impact of policy options, not as yieldingdefinitive predictions about the future.
Fundamental reviews of departmental models are necessary periodicallyto ensure that models remain relevant to the uses to which they areput and that there are no unfilled gaps in modelling work.
Models open to external scrutiny are better because outside feedbackhelps improve the quality of modelling. Contestability between modeloutputs and their underlying theory helps to build a climate in whichexcellence in modelling flourishes. An academic modelling panel shouldbe established to oversee this process and provide an audit facility forlong-term models.
The Economic and Social Research Council should expand its existingresearch programme to ensure that the UK becomes a centre of appliedmodelling expertise. Within Whitehall, a modelling network should beestablished to exchange best practice.
The case for a central modelling unit is unproven. But there is scopefor departments to work more closely together in developing modelsof mutual interest.
Improving the
quality of
microeconomic modelling
in Government
61
10.1 The conclusions in earlier Chaptersshould establish the conditions in whichanalysis can flourish. The present Chapter
considers the final piece in the jigsaw –how to improve the quality of modellingin Whitehall.
10.2 An important strand of the fieldworkfor the project has been to identify whatmicroeconomic modelling is done inGovernment, to form a view about howgood it is and how far it meets the needsof policy makers. The conclusion of thatexamination is that there is extensivemodelling work across Whitehall. Much isof good quality. But each model examinedin the case studies does have limitations.
10.3 The conclusions of this Chapter buildon best practice to help fulfil the visionoutlined in Chapter 4. The right models needto be available to Government and usedappropriately. The twin themes informing theconclusions are openness in thedevelopment of models and contestabilityof advice based on applied microeconomicmodelling.
Policy makers and analystsneed to use models intelligently10.4 Microeconomic models offer potentiallyimportant and valuable insights across arange of Government policies.
10.5 But models must be used intelligently.Analysts and policy makers should alwaysremember that:
• models are not the only tool in theanalyst’s armoury. As emphasised inChapter 9, basic data description alonecan offer powerful analytical insights.And new policies need to be fully thoughtthrough at a conceptual level before theyare modelled;
• models are only as good as the dataupon which they are based. One of thekey differences between the UK and theUS is a willingness of the US to invest indata collection for policy evaluation andappraisal which in turn permits moreextensive modelling in support of policymaking;
• models must not become ‘black boxes’either to the analysts who create themor to the policy customers who usetheir outputs;
• the limitations of models need to beclearly understood. Models do not giveanswers, they give insights. Models helpdefine the areas of doubt, but they cannever be definitive in all areas and theirresults should interpreted with care;
• the process of modelling itself is asimportant as the outputs of the modelsin informing policy. Developing a modelforces a rigorous understanding of marketsor systems as well as policy and its possibleimpacts (see Box 10.1).
Addi
ng It
Up
62
Box 10.1. Policy Insights from Modelling – Lord Chancellor’sDepartment
The Lord Chancellor’s Department runs a number of statistically based models forforecasting Legal Aid expenditure. Full details of one of these are contained in case study9. A variant of the methodology used in that model helped support the proposed movefrom an hourly based system of remuneration for solicitors to a ‘stage based’ system. Thischange is designed to reduce the potential incentive to solicitors to spin out the time spenton a case – so called ‘hour creep’. In the process of the detailed modelling work it becameclear that some cases lent themselves to being split into multiple cases to maximisesolicitor revenue. This set in train work to minimise this loophole.
Government needs to ensure it has the right models10.6 Government needs to checkperiodically that it has all the models that itneeds and only the models that it needs.
10.7 Long-term models can develop a lifethat is independent of the policy area theywere originally designed to support. TheDETR Household Formation model is onemodel whose role is being reassessed in thelight of current policy developments (seecase study 3 and Box 10.3). This closelyparallels the experience of the National RoadTraffic Forecasting Model, examined in casestudy 1. In that case a change in thephilosophy of the road building programmehas led to the development of a more flexiblepolicy tool.
Conclusion 33: departments should review their modelling capacity from first principlesto make sure that their modelling workremains appropriate to the current rangeof departmental objectives.
Greater openness and access to models will help10.8 To be useful to policy makers amicroeconomic model needs to be:
• soundly based in theory;
• statistically or econometrically robust;
• appropriate to the questions it is tryingto address.
10.9 These conditions are more likely to befulfilled if:
• models are more open to external scrutiny;
• models are more widely available for useoutside their department of origin;
Improving the
quality of
microeconomic modelling
in Government
63
Box 10.2. Where Are the Modelling Gaps?
The case studies have necessarily concentrated on those areas where models already exist.But identifying policy areas where modelling would be valuable but is not taking place isjust as important. This study has revealed several such areas of potential gaps in modelling:
• a well-founded econometrically based model of both internal and external migrationwould be of use to a number of departments notably Home Office, Foreign Office andDETR. It would also be of value in assessing the work of GAD and ONS in producingnational and sub national population projections;
• modelling in support of target setting, whether operational (in the case of the electronicdelivery of Government target) or PSAs;
• modelling of financial services;
• modelling of transport use other than roads and the interaction with road use.
There is no straightforward benchmark for determining what the priorities for modellingacross Whitehall should be. Without a detailed understanding of individual policy areasit is impossible to judge where the distribution of limited modelling resources should go.Instead the aim should be to open up departmental choices to peer review and to carryout analyses of modelling needs with policy makers. Doing so would help identify gapsand provide a mechanism for judging whether they should be filled.
• the development process involves outsiders.
10.10 The conclusions are not designed toensure that all Government models should beat the technical leading edge ofmicroeconomic modelling. Models need tobe ‘fit for purpose’. Sometimes it is indeedimportant for models to be state of the art(e.g. much if not all of the OR modelling forthe Centre for Defence Analysis falls into thiscategory). But often an informal “back of theenvelope” model will be all that the policyprocess requires. In general, there is a properdivision between pure research undertaken inuniversities, testing new or unproventechniques, and the need for defensibleestimates required in Government.
10.11 Openness improves modelling throughtwo channels:
• openness in outputs generates a richerand better informed policy debate. In thiscontext it is striking how the policy of theBank of England in publishing its own suiteof models and actively explaining their rolein interest rate policy making has raisedthe sophistication of that public debate(see Box 10.4). Publication of results frommicroeconomic models in Governmentwould have a similar effect;
• openness in operation and workingsencourages dialogue between Whitehalland the specialist community which helpsto enhance the standard of modelling.One of the lessons to learn from themodels of the water regulator (see Box10.5) is that greater transparency leadsto more robust modelling.
Addi
ng It
Up
64
Box 10.3. Proposed Revisions to the DETR Household Formation Model
The DETR Household Formation Projection Model was developed to underpin regional andlocal planning guidance for new housing provision. To date it has been a sophisticated, butpurely statistical, projection of past trends in demography and regional/local migration.There is no attempt within the model to evaluate the impact of alternative economicscenarios or policy options on household formation.
However, the departmental press release in March 1999, which published the latestprojections, contained, for the first time, a sensitivity analysis showing the impact onhousehold formation of different economic assumptions. And the department is currently inthe process of tendering for the development of a new model which will project householdformation as a function of economic as well as demographic variables and allow the impactof policy changes to be exemplified under a range of scenarios.
Box 10.4. Bank Of England Macroeconomic Models
The Bank of England has undertaken a policy of aggressive openness with itsmacroeconomic modelling. The full suite of models has been published in a book called‘Economic Models at the Bank of England’ (Bank of England 1999). The philosophyunderlying their use has been promoted through speeches by Bank officials (for example,John Vickers’ Speech to the Governors of NIESR, March 1999). The impact of the debate isseen in the sophisticated use of complex Bank of England data in public debate throughoutthe media.
10.12 Modellers in Whitehall have shown awidespread willingness to seek advice andexpertise from outside Government. Oftenthere are external models with whichGovernment models compete. The positionvaries between models:
• models can have private sectorcompetitors. The DTI energy model hasrivals run by private sector energyconsultants against which it has performedwell, historically;
• the academic sector presents challenge inother areas. For example, both theTreasury and the DSS run tax/benefitmodels. So too do the Institute for FiscalStudies (IFS) and the Department ofApplied Economics (DAE) in Cambridge.The IFS has recently audited the TreasuryInter Governmental Tax Benefit Model(IGOTM – see case study 11). Similarexercises in the past have also includedthe DSS Policy Simulation Model PSM(see case study 12);
• sometimes the relevant comparators areinternational – there are no private sectoranalogues to the DSS PENSIM model butthere are similar models in Canada (basedon a model developed at CornellUniversity) and Australia. The DTI energymodel also has competitors in internationalorganisations (the International EnergyAgency and the European Commission);
• the National Road Traffic Forecastingmodel has no rivals or direct comparators,but is open to external scrutiny andoverseen by a steering group ofstakeholders and experts. The results andworkings have been published in WorkingPapers and presented internationally. Butthere are no similar models elsewherebecause the UK has different data setsfrom other countries;
• the Home Office model of property crimeis still under development but earlyversions have been published with a viewto gathering feedback from the wideracademic community.
10.13 There would be gains in formalising thisopenness and establishing the presumptionthat long-term models should be publishedand externally audited.
Conclusion 34: a Modelling Academic Panel should be established to define andsupervise the process of publication andexternal review of long term models used inGovernment. This Panel should be chaired bythe Chief Economist in the Treasury PublicServices Directorate. It should include bothcivil servants and academics.
10.14 Microeconomic models in Whitehallneed to be readily available for others to use.
Improving the
quality of
microeconomic modelling
in Government
65
Box 10.5. OFWAT Efficiency Models (Case Study 2)
OFWAT have a suite of 17 models which are used for calculating the relative efficiency ofwater companies as part of the price setting process. Outside scrutiny is intense. The watercompanies have a powerful incentive to test the limits of OFWAT’s models. The Regulatorknows that water companies can seek an investigation by the Competition Commission orultimately judicial review.
As a result OFWAT has consulted widely in the development of the models. The originalsuite was developed in association with academics at the University of Warwick.Throughout the process the models have been well documented and open to publicscrutiny to secure feedback and encourage collaboration. As a result of this transparencythe models are defensible in the public domain.
10.15 The experience of shared models inWhitehall is limited. It is generally the case,with the exception of IGOTM, that modelsare developed and understood in only onedepartment, even where their outputs areused in others. Technological advance hasmade open access to models easier. The IFSTax Benefit model (TAXBEN) and the Warwickmacroeconomic model can be run over theWorld Wide Web; the DAE’s model for taxbenefit calculations (POLIMOD) is availablein the House of Commons library. Portabilityfits well with the wider aspirations oftransparency outside Government and greaterawareness of modelling within Government.
10.16 As one of its first tasks, the ModellingAcademic Panel should consider ways to helpdepartments to make their models ‘portable’so that others within and outsideGovernment can use them.
10.17 Ex post audit and publication of modelsis not sufficient. Outside expertise needs tobe involved from the outset in thedevelopment of new models.
10.18 There is a range of ways of accessingexternal advice:
• an external steering group is a minimalistapproach to obtaining outside input,although closer technical collaborationwith a subset of members can providecloser engagement;
• secondment of Government analysts toacademic institutions in order to buildspecific models (see Box 10.6) allowsexternal input to the modelling process;
• full contracting out represents the mostcomplete use of the external world, butneeds careful management. For a fullerdiscussion see Chapter 11.
Conclusion 35: outside expertise should be involved in model development from theoutset through an external steering group orother mechanisms.
Box 10.6. Reaching Out
Close Engagement with Academics
One model of closer engagement is thatused by the Department of Health in thedevelopment of its Long-Term Care ofthe Elderly Finance Model. In this caseDH seconded an analyst one day a weekto the London School of Economics (LSE)to work specifically on the model. Thisapproach had the effect of both shieldingthe modelling work from the immediatepressures of being in the office andgenerated close contact with academicsworking in this and related fields. Thishas the advantage of ensuring fulldepartmental ownership of the workingsand outputs of the model in the future.
Modelling “Clinics” In EconomicConsultancies
Both Frontier Economics and LondonEconomics, two private sector economicconsultancies involved in the supply ofanalysis and modelling to governmentdepartments, obtain advice on thetechnical aspects of modelling fromacademics. In each case, they hold‘econometric surgeries’. These operatein the same way as GP appointments.Consultants are able to bookappointments with a Professor ofEconometrics who is available at specifictimes to discuss any problems they mayhave with their current project.
Addi
ng It
Up
66
Creating a wider culture in which modelling flourishes10.19 Openness in Government modellingneeds to be complemented by anenvironment where applied microeconomicmodelling is widely taught and understood.Openness has no advantages if there is notthe expertise in the outside world to providea challenge to Government analysis.
10.20 Among the existing Economic andSocial Research Council (ESRC) centres ofresearch, only the IFS explicitly includes usingmicroeconomic and microeconometrictechniques in its outline objectives. TheESRC’s macroeconomic modelling researchprogramme, which has recently come to anend, is widely regarded as having pushed theUK to the forefront of technical modelling.A similar programme in microeconomicmodelling would have the advantages of:
• establishing a bigger pool of experiencedmodellers;
• promoting more rapid technical advances;
• enhancing the policy debate throughgreater competition between models(“contestability”);
• creating a larger constituency for improveddata.
10.21 An ESRC programme would generatethe microeconomic modelling, datamanipulation skills and in-depth knowledgeof large data sets which are all in shortsupply both in Government and in theacademic sector. The narrowness of this seamof talent has three broad causes:
• the declining numbers studying economics atboth undergraduate and postgraduate level;
• the bias against applied economic workestablished through the Higher EducationFunding Council’s Research AssessmentExercise which pushes talented studentsaway from applied microeconomics;
• a lack of interchange opportunities. Abroader microeconomic profession wouldopen opportunities for greater interchangebetween the academic and governmentalsectors. Experience, highlighted in thedevelopment of models in the US and incase study 7 on long-term care, shows thatthis is probably the most effective way ofspreading best practice between the twosectors.
10.22 Such a programme would alsoimprove the quality of modelling where thisis presently being held back through lack oftheoretical understanding. One commontheme arising in relation to most modelsacross Government has been the difficultyof modelling the behavioural implications ofpolicy. Progress towards greater behaviouralmodelling is still at rudimentary stage in theoutside world, but a concerted effort couldhelp bring to fruition developments whichseemed promising in the 1980s. Such astirring seems to be happening in the USwhere dynamic microsimulation models, withsome behavioural elements, are becomingmore widely used within Government.
10.23 Additional external centres of expertisein modelling would help keep Whitehallmodellers on their toes. Evidence fromCanada and the US suggests competitionbetween models is a force for theirimprovement. In addition, greater attentionpaid to microeconomic modelling leads toa higher quality policy debate.
10.24 Additional modelling in academia andthink tanks would also increase the demandfor data. Modelling depends on data and thegreater the demand for better and morereadily available data, the greater theincentive to collect them.
Improving the
quality of
microeconomic modelling
in Government
67
Conclusion 36: the Chief Economist in the Treasury Public Services Directorate shouldencourage the ESRC to adopt the aim ofmaking the UK a centre of internationalexcellence in applied microeconomicanalysis by 2005.
Pooling knowledge10.25 There is a case for strengthening linksbetween modellers within Whitehall throughthe development of an informal technicalnetwork of specialists, supported perhaps byuse of the Government secure intranet (GSI).(Some technical level links do already existwithin the Operational Research andstatistical communities.)
10.26 Such links would help to enhancethe quality of modelling in Governmentin a number of ways:
• modelling within departments is done bystatisticians, economists and operationalresearchers. Exchanging information andgood practice between specialists wouldhelp break down professional demarcationlines and raise the standards of modelling.In particular, establishing clearer linksbetween professions might help with theanalysis (if not the formal modelling) ofbehavioural changes, where theoperational and social researchcommunities have also been active;
• they would allow techniques andknowledge to be shared more effectively;
• a cross-departmental network might beone channel through which externaltechnical expertise is accessed informally;
• better cross-departmental linkages of thissort could provide the infrastructure forthe support of specialist modellers. Greaterknowledge and familiarity with activity inother government departments could helpthe flow of modelling talent betweendepartments.
Conclusion 37: a modelling network chaired by the Chief Economist in theTreasury Public Service Directorate should beestablished by March 2000 for specialists inGovernment to promote exchange of goodpractice at technical level and to signal theimportance of modelling within Government.
10.27 There is a need for more jointdevelopment of models between Whitehalldepartments.
10.28 The Government’s objective settingframework increasingly emphasises the needfor cross-cutting objectives with aconsequent demand for cross-cuttinganalysis, and within that, joint modelling.Box 10.7 highlights this in relation totax/benefit modelling. Other areas wherecross-departmental modelling might enhancejoint understanding of policy issues includevehicle CO2 emissions and company taxation.As in the case of portability of models,technology now makes more feasible thejoint development of models with a viewto their being used independently withindepartments.
Conclusion 38: the review of analysis to be conducted by the CORE team in HM Treasury(see conclusion 4) should address explicitlythe need for and provision of jointlydeveloped microeconomic models.
The case for a central modelling unit is not proven10.29 Locating modelling capacity away fromdirect departmental responsibility has someattractions:
• modelling is a specialist business and thereis much that can be learned from sharingcommon experience;
Addi
ng It
Up
68
• a central modelling unit could be ring-fenced from the day to day concernswhich can often crowd out developmentalwork on economic models. Several long-term models in Government have gonethrough periods of neglect as analystshave been assigned to other work.A central unit might offer shelter fromsuch pressures;
• the head of such a unit might be givenresponsibility to report directly toParliament (as is the case with theGovernment Actuary at present), therebyenhancing the perceived professionalism,integrity and independence of advice;
• a career in a central unit, perhaps withstronger links to the academic sector,might offer an alternative career path tothat of the ‘generalist economist’ indepartments which might be moreattractive to those with particularmodelling skills. It might also provide anenvironment to address the problems ofdeveloping plausible career paths forspecialists in this field.
10.30 However, the counter arguments aremore compelling:
• experience with the provision of analysisfrom existing central units – such as GADand ONS – is mixed. Remoteness from thepolicy process can mean a loss of feel forthe concerns of policy among specialistswhile policy makers can lack anappreciation of the limitations ofmodelling;
• there is evidence that ring-fencing can beachieved within departments if sufficientpriority is given to modelling work. This isthe approach which has been used in DSSwith the establishment of a dedicatedmodelling division in the AnalyticalServices Division to carry forwarddevelopmental work.
10.31 On balance, while there are realspecialist skills in manipulating complex datasets which it might be both economical andpractical to centralise, the need for policy-relevant and timely modelling in departmentswill probably always militate against a centralmodelling unit.
Improving the
quality of
microeconomic modelling
in Government
69
Box 10.7. Joint Working in Tax/Benefit Modelling – a Case forRevisiting?
The interaction of the tax and benefit system is an area of clear policy overlap, with severaldepartments having an operational and policy interest in the area. This joint analytical needwas recognised in the late 1980s with the development of the Inter Governmental TaxBenefit Model (IGOTM). At the time it proved impossible to meet the needs of all interesteddepartments, and as a consequence DSS retained its own model (Policy Simulation Model –PSM).
Policy developments now mean that the Government is channelling more of its support forthose on low incomes through the tax system. As a result departments are having to learn newskills and gear themselves up to analyse and implement new policy options. In thisenvironment there may be a case for a new interdepartmental group to examine the feasibilityof developing a new comprehensive tax benefit model to be used by all departments.
11.1 The conclusions in Chapters 5-10should put in place high quality analysis andmodelling in Whitehall. But Governmentdoes not have a monopoly on wisdom.It should take advantage of the work ofacademics and think tanks to confirm itsown analysis and to supplement it wherenecessary. The existence of these strong linksbetween Government and the outside worldis one of the key differences between theexperience of the US and the UK in the useof analysis.
Strengthening links between Government and academia11.2 Government has to be able tocommunicate its needs and priorities clearlyto the academic sector if the research agendain academia is to reflect its requirements.It can do so through:
11. MAKING BETTER USE OF THE EXTERNAL WORLD
Summary
Government should make better use of links to the academic world.Academic research can bear on practical policy questions. One of the rolesof specialists in Government is to interpret the significance of academicdebate for policy-makers and Ministers.
There are a number of ways in which departments can influence and drawon academic work. These include effective use of departmental researchbudgets; the funding of applied research through the payment of“summer money” as in the US; and the secondment into the Civil Serviceof internationally recognised experts.
Getting the right balance between internal and external work will varyfrom one policy area and specialism to another. More contracting-out formodelling and other analytical work may be required in the future if itbecomes increasingly difficult to attract sufficient high quality analystsinto government service. But Government will always need to retain somespecialist capacity to manage external projects and, more critically, to actas an intelligent customer.
Addi
ng It
Up
70
• direct access to the academic researchagenda through its representation on theEconomic and Social Research Council(ESRC). The mission statementunderpinning the ESRC’s ThematicPriorities exercise in 1995 had support forpolicy relevant research as its foundation6;
• departmental research budgets – and thepublication of research strategies. Theexample of DfEE has been discussed inChapter 6;
• more core funding of research institutes.The UK cannot aspire to follow preciselythe example of the US in the variety andquantity of its independent researchinstitutes. However, redirection of researchbudgets towards core funding for researchinstitutes may go some way towardsenriching the environment for policydevelopment.
11.3 Three more specific practical ideasdrawn from the US could also strengthenlinks with academia:
• a ‘Summer Money’ scheme. Through thisacademics are paid an additional twoninths of their salary by Governmentdepartments to work on specific researchprojects of direct relevance to Government.The benefit is that the academics also tendto gear their ongoing research programmesto similar topics, establishing a direct linkto the academic research agenda. There isa strong case for piloting a summer moneyscheme in the UK;
• interchange promoted through the InterGovernmental Personnel Act under which afixed number of academics are secondedto Government for a year to work jointlyon specific research projects withGovernment analysts;
• use of experts to run departmental researchprogrammes. The US Labour Departmentemploys an outsider (not a politicalappointee), usually an eminent laboureconomist, as its director of researchon a rotating basis. There may be scopefor something similar in the UK.
Conclusion 39: a ‘challenge fund’ of £50,000 (administered by the CORE team inthe Treasury) should be established, from July2000, to fund five academic summerplacements in Whitehall each year.
Conclusion 40: the Civil Service reform plans to identify 100 key tasks to be carriedout by private sector secondees shouldinclude a number of placements forinternationally recognised specialists.
11.4 As well as ensuring that academicresearch is available, Government needs tobe able to ensure its dissemination and usein the policy process. This has two aspects:
• analysts in Government need to beup to date with the academic debateon contentious issues that bear on policydevelopment and interpret these for policymakers and Ministers. Departments do notalways have the luxury of waiting for anacademic consensus to be reached beforeformulating policy. Where a pragmaticjudgement about the way forward needsto be taken – for example, in measuresto reduce structural unemployment – thepolicy process must, so far as possible,be informed by an understanding ofacademic debate. These issues areexplored in more detail in case study 10on the development of policy on thelabour market;
Making better
use of
the external
world
71
6 The statement said that the mission is to “fund high quality research and training which meets the needs of users and enhances the
UK’s competitiveness, quality of life, and the effectiveness of public services and policy.”
• the creation of Whitehall knowledge pools.The CMPS is taking forward theGovernment strategy for knowledgemanagement. Departments may also needto prepare strategies. For example, theDfEE has developed a strategy that isgeared around building an accessible bodyof research in house. And the CochraneCentre for Medical Research in DH alreadydoes this for medicines. Turning theserepositories of information into valuabletools will require management. Thesemoves by individual departments shouldform stepping stones towards the morewide-ranging and ambitious CMPSaspiration to establish a Whitehall wideknowledge pool.
Strengthening links betweenGovernment and the private sector11.5 There may be scope for furthercollaboration with the private sector. Thisapplies to both data and modelling:
• few references to private sector data sourcesemerged from departments’ responses tothe project questionnaire or from work onthe case studies. However, some instanceswhere data are collected from the privatesector do exist. For example, a 5% sampleof mortgages is collected from mortgagelenders and provides useful informationabout house prices. There may be scopefor departments to do more of this.For example, individual water companieshave detailed information about theimpact of water metering on water use.This would be useful evidence to havein the formulation of domestic watercharging policies;
• there may be areas of overlapping interestthat could be exploited more. Forexample, modelling of the costs andbenefits of contraception by Durex wasvaluable in work by the Social ExclusionUnit on teenage pregnancy. Butopportunities may be limited. There willinevitably be some areas where overlapsin modelling between the private andthe public sector are non-existent.No company, for example, attemptsto estimate the future population inEngland and Wales.
Conclusion 41: departments should explore with the private sector the feasibilityof accessing and using private sector datasources.
The balance between externaland internal analysis11.6 There is an important role forcontracted-out analytical work but it needsto be carefully defined and managed.
11.7 The advantages of contracting-out are:
• it allows one-off projects to be carried outwithout a permanent expansion of staffnumbers;
• it gives access to specialist expertise;
• it draws on the increasing range ofacademic and private sector organisationsthat can undertake such work.
11.8 There are also potential disadvantages:
• contracting-out may offer less good valuefor money than work done in house whereanalysts need to be close to the policyprocess;
Addi
ng It
Up
72
• knowledge transfer – which should be theaim when work is contracted-out – may bedifficult to achieve. This is more likely to bethe case where the work contracted-out isthe development of a sophisticated modelsuch as PENSIM, than in the case, say, ofstudies to evaluate policy implementationor basic scientific or social research.
11.9 The balance of work contracted-outtends (unsurprisingly) to vary significantlybetween specialist disciplines. Box 11.1describes some experiences of contracting-out analytical work in the data and modellingspheres.
11.10 Overall, contracting-out is likelyto play an increasingly important role,particularly if it becomes more difficultto recruit high quality specialists to workin Government. Government will, as aminimum, need the specialist capacity toexploit it fully and to act as an intelligentcustomer in commissioning and managingcontracted-out work.
Making better
use of
the external
world
73
Box 11.1. Contracting Out – a Mixture of Experiences
Data
• The Social Exclusion Unit (SEU) commissioned Southampton University to undertake,at short notice, an analysis of teenage pregnancy using the NCDS when Governmentdepartments were unable to supply information in this field.
• Basic data cleaning and reconciliation is contracted out with success by the USGovernment.
Models
• Successes in the UK include DETR’s contract for maintaining and developing modelsof Household Formation both with Anglia Polytechnic University and the Departmentof Applied Economics at Cambridge.
• Some departments of the US Government use a tax benefit model supplied by theUrban Institute (a social think tank).
• Australian Government departments buy in source code for models developed by theNational Centre for Social and Economic Modelling at the University of Canberra andthen manipulate it themselves.
• A comparative failure in both the UK and Australia was the contracting-out pensionsmodelling. In neither case did this lead to efficient knowledge transfer to Government.
12.1 The success of this report dependson the implementation of its conclusions.However, as the report suggests, improvingthe demand for and supply of analysis andmodelling is a complex issue. There is noone proposal that would lead to betteranalysis and no one agent of change. It istherefore crucial that clear roles are allocatedto the key players responsible for each clusterof conclusions.
12.2 This Chapter therefore:
• identifies the main players responsiblefor key strands of conclusions;
• describes how the report’s findings will bedisseminated to analysts and policy makersthroughout Whitehall;
• assigns a responsibility to key players foreach individual conclusion and a timetablefor taking action.
12. IMPLEMENTATION STRATEGY
Summary
This report will lack value unless its conclusions are implemented.
Key players for the implementation are CMPS (responsible fortraining and the dissemination of best practice), Permanent Secretaries(responsible for the supply of analysis in departments) and the Treasury(responsible for looking more closely at departmental analytical plans inthe context of the SR and PSA processes, and, together with the CabinetOffice, for a “challenge” role vis-à-vis departments).
Implementation will be overseen by an Implementation Group(with a membership drawn from the Project Steering Group).This will meet periodically to assess progress with implementation.
A table summarises the conclusions in the report and gives a preliminaryestimate of cost. This will need to be further refined by the ImplementationGroup. The Group will also need to commission further work on theprioritisation of the report’s conclusions.
Addi
ng It
Up
74
A range of players will haveresponsibility for implementation12.3 The key players involved inimplementation of the reports’ findingswill be:
• an Implementation Group chaired by theHead of GES. The Steering Group for thisproject has agreed to meet twice a year(and perhaps more often initially) tooversee implementation and checkprogress against the proposed timetable.The Implementation Group will beserviced by members of the CORE teamin the Treasury;
• CMPS, which will be responsible forspreading best practice and training forMinisters and senior officials. They will alsobe involved in bringing togetherinformation relevant to policy formulation,from both within and outside Whitehall;
• Permanent Secretaries, who will becharged with implementation of theconclusions addressed to departments,within the context of the ModernisingGovernment agenda;
• the Treasury, whose role will be: to ensuresound analysis supports the SpendingReview and PSA process; to identify cross-cutting issues and make sure they aresupported by analysis; to administer theseed-corn fund for cross-cutting analysis;and to service the Implementation Groupand the modelling Academic Panel;
• Heads of Profession and Chairs ofManagement Groups who will play animportant role in overseeing developmentsacross their own professional domains,challenging the distribution of analyticalcapabilities within and across departments(where appropriate), promoting betterjoint working between professionals andensuring that high quality specialist staffare available across departments;
• ONS and the National Statistician whowill be involved in ensuring, whereappropriate, that the initiatives beingundertaken through the GSS arecommunicated to all specialists involvedin analysis and modelling.
The findings of this study needto be widely disseminated throughout Whitehall12.4 Dissemination of the report’sconclusions will form an importantcomponent of implementation. Spreadingawareness of the conclusions will help toensure that the importance of analysis andmodelling is instilled in the consciousness ofofficials and Ministers at the highest level.
Timetable for Implementation12.5 Table 12.1 assigns responsibilities foreach of the conclusions to the key players,as well as a date by which the conclusionshould be acted on.
Implementation strategy
75
Costs...12.6 Table 12.1 gives a preliminary costingand assessment of the impact ondepartments for each of the detailedconclusions.
12.7 The most expensive are likely to be: theestablishment of a seed-corn fund; additionaltraining for policy-makers in analysis; and,if accepted, increased pay for analysts inWhitehall. If the proposals, whenimplemented, lead to the demand for agreater analytical capacity across Whitehallor for new data sources, that too would leadto additional costs.
12.8 Implementation of some of theconclusions might lead to increasedadministrative effort on the part of theTreasury CORE team or their analogues indepartments. The preliminary costingspropose an additional 2/3 staff for theTreasury CORE team.
12.9 Finally, a number of conclusions inTable 12.1 are allocated to the CMPS to takeforward in the context of civil service reform.These proposals have not been separatelycosted as they will form an integral part ofother programmes. It should be noted,however, that funding plans have not yetbeen agreed for those programmes. Theremit of the CMPS has expanded since theoriginal plans for the organisation weredrawn up.
12.10 Further work will be required torefine the costings. The ImplementationGroup will also need to commission morework on the prioritisation of the conclusionsin this report.
...and benefits12.11 The benefits of improved analysisare also hard to quantify.
12.12 But the case studies make clear thescale of potential sums at stake in decisionssupported by analysis. For example:
• revenue lost through smuggling oftobacco is presently £2.5 billion pa;
• population modelling helps determinethe affordability of long-term pensionprovision. The original State EarningsRelated Pension Scheme introduced inthe mid-1970s became increasingly lessaffordable as the size of the likely rise independency ratios in the early part of thenext century became clearer. In a series ofreforms in the 1980s, the value of SERPSbenefits was reduced.
12.13 Sometimes the value of analysis isthat it can help change a Ministerial decision.For example, within weeks of the 1997General Election, the new administrationcalled a Water Summit and subsequentlyembarked on a review of water charging.One of the policy options considered wasmoving from a system of domesticunmeasured charges based on rateablevalues to a system based on Council Taxbands. Contrary to the initial expectations ofMinisters, analysis of the proposal revealedthat such a system would be more regressivethan the existing arrangement.
12.14 Even when analysis does not alterMinisters’ decisions, it may nonetheless havea value. For example, analysis of theinformation about take-up gathered frompilots for the New Deal for the disabled didnot affect Ministers’ decision to move onto national roll-out. In such a case, Ministerswill have a clear idea how a policy will workat an early stage of its implementation.Such information has a value, albeit onethat is hard to quantify.
Addi
ng It
Up
76
Implementation strategy
77
Imp
act
in D
epts
Ad
dit
ion
alR
eso
urc
esR
equ
ired
By
Wh
enB
y W
ho
mC
on
clu
sio
n
£3-4
m
Part
of
exp
ande
d C
ORE
role
FY 2
000/
01H
MT
Co
ncl
usio
n 6
:a
seed
-cor
n an
alys
is a
nd m
odel
ling
fund
sho
uld
be e
stab
lishe
d fr
om t
he
finan
cial
yea
r 20
00/2
001.
Thi
s w
ould
be
subj
ect
to a
suc
cess
ful b
id o
n th
e Re
serv
e fo
r
2000
/200
1 an
d in
the
pre
sent
Sp
endi
ng R
evie
w f
or f
utur
e ye
ars.
The
fun
d sh
ould
be
adm
inis
tere
d by
the
Tre
asur
y w
ith b
ids
cons
ider
ed b
y a
com
mitt
ee d
raw
n fr
om d
iffer
ent
spec
ialis
ms
and
diffe
rent
dep
artm
ents
.
Build
s on
exi
stin
g p
eer
revi
ew p
lans
To f
orm
par
t of
CM
PS
fund
ing.
In e
xist
ing
pee
r re
view
timet
able
CM
PSC
on
clus
ion
5:
CM
PS le
d p
eer
revi
ews
of b
usin
ess
pla
nnin
g an
d p
olic
y m
akin
g sh
ould
exam
ine
the
qua
lity
of a
naly
sis
and
mod
ellin
g w
ith a
vie
w t
o sp
read
ing
best
pra
ctic
e an
d
stim
ulat
ing
imp
rove
men
t.
Nee
d to
defi
ne D
ept
anal
ytic
al s
trat
egy
and
liais
e w
ith C
ORE
. Bu
ilds
on e
xist
ing
busi
ness
pla
nnin
g p
roce
dure
s
Incr
ease
in s
taffi
ng o
f
CO
RE t
eam
(e.
g. 1
new
G5
and
2 ne
w G
7s)
in
HM
T. T
his
role
to
be
par
t of
the
job
of t
his
new
gro
up.
Firs
t re
por
t 20
01H
MT
Co
ncl
usio
n 4
:th
e C
ORE
tea
m in
HM
Tre
asur
y sh
ould
rev
iew
ann
ually
dep
artm
enta
l pla
ns
for
anal
ysis
and
res
earc
h an
d p
rese
nt a
rep
ort
sett
ing
out
the
mai
n fin
ding
s to
the
Imp
lem
enta
tion
Gro
up c
harg
ed w
ith o
vers
eein
g th
e im
ple
men
tatio
n of
thi
s re
por
t.
Dep
endi
ng o
n
outc
ome
of r
evie
w,
shou
ld le
ad t
o m
ore
cohe
renc
e be
twee
n
imp
act
asse
ssm
ents
Dep
ende
nt o
n
outc
ome
of r
evie
w
Dep
ende
nt o
n
outc
ome
of r
evie
w
Dep
ende
nt o
n
outc
ome
of r
evie
w
Co
ncl
usio
n 3
:su
bjec
t to
the
out
com
e of
the
pre
sent
rev
iew
, de
par
tmen
ts s
houl
d ca
rry
out
and
pub
lish
a si
ngle
com
pre
hens
ive
imp
act
asse
ssm
ent
of a
ll ne
w p
olic
ies,
pro
gram
mes
and
pro
ject
s.
To f
orm
par
t of
CM
PS
fund
ing.
Acc
ordi
ng t
o ex
istin
g
CM
PS s
ched
ule
CM
PSC
on
clus
ion
2:
CM
PS s
houl
d in
clud
e gu
idan
ce o
n th
e us
e of
ana
lysi
s in
bes
t p
ract
ice
pro
toco
ls f
or c
ircul
atio
n to
Min
iste
rs a
nd o
ffici
als.
Suc
h gu
idan
ce s
houl
d he
lp d
isse
min
ate
“Gre
en B
ook”
prin
cip
les
in n
on-t
echn
ical
ter
ms
(e.g
. th
roug
h th
e us
e of
wor
ked
exam
ple
s)
as w
ell a
s th
e fin
ding
s of
NA
O v
fm s
tudi
es.
To f
orm
par
t of
CM
PS
fund
ing.
June
200
0C
MPS
Co
ncl
usio
n 1
:tr
aini
ng c
ours
es f
or M
inis
ters
and
sen
ior
offic
ials
run
by
the
CM
PS s
houl
d in
futu
re in
clud
e a
sess
ion
on t
he im
por
tanc
e of
goo
d an
alys
is f
or t
he p
olic
y-m
akin
g an
d
deci
sion
-mak
ing
pro
cess
es.
Tab
le 1
2.1
: Su
mm
ary
of
con
clu
sio
ns
Addi
ng It
Up
78
Imp
act
in D
epts
Ad
dit
ion
alR
eso
urc
esR
equ
ired
By
Wh
enB
y W
ho
mC
on
clu
sio
n
1 G
7 ye
ar e
ach
Parli
amen
t p
er d
ept
(£50
,000
eve
ry fi
ve
year
s)
No
cent
ral r
esou
rces
Revi
ew d
urin
g cu
rren
t
Parli
amen
t on
a c
ycle
app
rop
riate
to
dep
t
busi
ness
pla
nnin
g
All
Perm
Sec
sC
on
clus
ion
10:
dep
artm
ents
sho
uld:
•re
view
pol
icy
on b
eddi
ng o
ut o
nce
per
Par
liam
ent
with
a v
iew
to
mai
ntai
ning
the
rig
ht
bala
nce
betw
een
cent
ral e
xper
tise
and
bedd
ed-o
ut e
ffect
iven
ess;
•en
sure
tha
t w
here
ana
lyst
s co
ntin
ue t
o fo
rm p
art
of c
entr
al s
pec
ialis
t un
its,
they
als
o
form
an
inte
gral
par
t of
the
wor
k of
pol
icy
team
s; a
nd t
hat
whe
re a
naly
sts
are
bedd
ed
out,
ap
pro
pria
te p
rofe
ssio
nal t
rain
ing
and
deve
lop
men
t p
rogr
amm
es a
re in
pla
ce.
To b
e in
corp
orat
ed in
exis
ting
pla
ns t
o
imp
rove
bus
ines
s
Part
of
Civ
il Se
rvic
e
refo
rm
As
per
imp
lem
enta
tion
of b
ette
r bu
sine
ss
pla
nnin
g in
Civ
il
Serv
ice
refo
rm p
roce
ss
All
Perm
Sec
sC
on
clus
ion
9:
as p
art
of t
he “
bett
er b
usin
ess
pla
nnin
g” s
tran
d of
the
civ
il se
rvic
e re
form
pro
cess
, an
alyt
ical
str
ateg
ies
shou
ld b
e de
velo
ped
by
gove
rnm
ent
dep
artm
ents
.
Revi
ew t
o ta
ke 0
.3 G
7
year
. C
ost:
£12
,500
Resu
lt: u
p t
o 6
new
G7
spec
ialis
t p
osts
(e.g
.3
each
in C
O &
HM
T).
Cos
t: £
300,
000
per
ann
um
June
200
0H
MT
CO
Co
ncl
usio
ns
8:th
e C
abin
et O
ffice
and
the
Tre
asur
y sh
ould
car
ry o
ut a
rev
iew
of
the
role
,
num
bers
and
typ
es o
f sp
ecia
lists
the
y ne
ed t
o su
pp
ort
an e
nhan
ced
“cha
lleng
e” f
unct
ion
to p
rom
ote
bett
er p
lann
ing
of a
naly
sis
by d
epar
tmen
ts b
y en
d Ju
ne 2
000.
Incr
ease
wor
k on
Spen
ding
Rev
iew
, by
0.5
G7
year
(£2
5,00
0
per
dep
t)
Smal
l mar
gina
l cos
t on
exis
ting
pub
licat
ion
of
PSA
s
In c
urre
nt S
pen
ding
Revi
ew
HM
T G
EP t
eam
Dep
artm
enta
l PFO
s
Co
ncl
usio
n 7
:th
e Tr
easu
ry s
houl
d us
e th
e SR
and
PSA
pro
cess
es t
o su
pp
ort
and
pro
mot
e
the
use
of a
naly
sis
and
mod
ellin
g th
roug
h: (
i) is
suin
g gu
idan
ce t
o de
par
tmen
ts a
nd (
ii)
pub
licat
ion
of a
naly
tical
wor
k un
derp
inni
ng P
SA t
arge
ts.
Implementation strategy
79
Imp
act
in D
epts
Ad
dit
ion
alR
eso
urc
esR
equ
ired
By
Wh
enB
y W
ho
mC
on
clu
sio
n
Any
cha
nges
in
per
sonn
el D
epar
tmen
ts
to b
e p
art
of t
he C
ivil
Serv
ice
refo
rm p
roce
ss
Subs
titut
ion
for
exis
ting
recr
uitm
ent
pro
cedu
res
2001
Dep
ts a
nd s
pec
ialis
t
man
agem
ent
grou
ps
Co
ncl
usio
n 1
5:de
par
tmen
ts a
nd p
rofe
ssio
nal m
anag
emen
t gr
oup
s sh
ould
rec
ruit
mor
e
exte
rnal
ap
plic
ants
dire
ctly
into
sp
ecia
list
(and
par
ticul
arly
eco
nom
ist)
pos
ts,
both
at
Gra
de
7 le
vel a
nd a
bove
.
Not
add
ition
al t
o
allo
catio
n fo
r ex
istin
g
recr
uitm
ent
com
petit
ion
Mod
aliti
es t
o be
revi
ewed
for
op
erat
ion
by 2
001
com
pet
ition
EGM
UC
on
clus
ion
14:
GES
sho
uld
recr
uit
som
e p
eop
le w
ho h
ave
a st
rong
ap
titud
e fo
r
qua
ntita
tive
met
hods
sui
tabl
e fo
r so
me
Gra
de 7
pos
ts b
ut,
who
do
not
nece
ssar
ily h
ave
the
all-r
ound
pot
entia
l to
reac
h G
rade
5.
Min
imal
cos
tA
t la
test
by
end
2000
Spec
ialis
t M
anag
emen
t
Gro
ups
Co
ncl
usio
n 1
3:th
ere
shou
ld b
e p
erio
dic
join
t m
eetin
gs o
f H
eads
of
Prof
essi
on.
Hea
ds o
f
Prof
essi
on s
houl
d co
nsid
er t
he c
ase
for
occa
sion
al jo
int
conf
eren
ces
betw
een
spec
ialis
ts
grou
ps
(i.e.
GSS
, G
ORS
, G
ES,
GSR
).
Incr
emen
t to
nor
mal
trai
ning
obl
igat
ion
Tota
l cos
t ar
ound
£80
0
per
ana
lyst
tra
ined
(ass
umin
g 4
days
trai
ning
).
Max
imum
of
50 a
yea
r
for
thre
e ye
ars
=
£40,
000
per
yea
r.
End
2000
Dep
ts/C
MPS
Co
ncl
usio
n 1
2:ne
w f
orm
s of
tra
inin
g fo
r sp
ecia
lists
sho
uld
be d
evel
oped
to
sup
por
t
join
t w
orki
ng (
e.g.
cou
rses
on
“par
tner
ship
wor
king
”) a
nd in
trod
uced
by
the
CM
PS f
rom
end
2000
.
Trai
ning
Rev
iew
in e
ach
Dep
t co
st 1
G7
year
(£50
,000
) in
add
ition
to in
crea
se in
tra
inin
g
budg
et
Tota
l cos
t of
the
reg
ion
of £
2,00
0 p
er h
ead
per
year
(ba
sed
on H
MT
figur
es).
But
may
not
all b
e ad
ditio
nal t
o
exis
ting
trai
ning
budg
et.
FCO
est
imat
e
£380
,000
eac
h ye
ar f
or
thre
e ye
ars
to ‘s
kill
up’
exis
ting
wor
kfor
ce.
End
2001
Ap
ril 2
001
All
dep
tsC
on
clus
ion
11:
•ot
her
dep
artm
ents
sho
uld
put
in p
lace
tra
inin
g ar
rang
emen
ts f
or p
olic
ymak
ers
sim
ilar
to
thos
e of
the
Tre
asur
y by
end
200
1;
•de
mon
stra
ted
pro
ficie
ncy
in a
naly
sis
shou
ld b
ecom
e a
req
uire
men
t of
pro
mot
ion
to t
he
SCS
from
Ap
ril 2
001.
Addi
ng It
Up
80
Imp
act
in D
epts
Ad
dit
ion
alR
eso
urc
esR
equ
ired
By
Wh
enB
y W
ho
mC
on
clu
sio
n
No
addi
tiona
l res
ourc
es
over
tho
se in
volv
ed in
man
agin
g th
e
Mod
ellin
g N
etw
ork.
Thro
ugho
ut 2
000
ON
S,C
MPS
, M
odel
ling
Net
wor
k
Co
ncl
usio
n 2
3:by
the
end
of
2000
and
per
iodi
cally
the
reaf
ter,
ON
S sh
ould
pro
mul
gate
wid
ely
the
reco
mm
enda
tions
of
the
GSS
tas
k fo
rce
on m
atch
ing
data
set
s to
all s
pec
ialis
ts
invo
lved
in m
odel
ling
wor
k an
d da
ta a
naly
sis,
thr
ough
wor
ksho
ps
and
sem
inar
s.
Ong
oing
pro
duct
ion
of
data
for
pub
licat
ion.
Shou
ld b
e in
crem
enta
l
to e
xist
ing
pro
cedu
res
Cos
t es
timat
es t
o be
deve
lop
ed b
ased
on
pub
licat
ions
of
othe
r
data
ser
ies
To fi
t in
with
dat
a
colle
ctio
n cy
cle
Rele
vant
dep
tsC
on
clus
ion
22:
dep
artm
ents
col
lect
ing
adm
inis
trat
ive
data
sho
uld
cons
ider
rel
easi
ng
seco
ndar
y so
urce
s ba
sed
on it
.
Non
e re
qui
red
Ove
r ne
xt 1
2 m
onth
sO
NS/
SEU
Co
ncl
usio
n 2
1:th
e O
NS
and
SEU
sho
uld
diss
emin
ate
mor
e w
idel
y th
eir
wor
k on
adm
inis
trat
ive
data
to
spec
ialis
ts,
for
exam
ple
at
the
GES
con
fere
nce
or t
hrou
gh t
he
pro
pos
ed M
odel
ling
Net
wor
k (s
ee C
hap
ter
10),
by
the
end
of 2
000.
Like
ly t
o be
low
thou
sand
s of
pou
nds
thro
ugh
addi
tiona
l
com
put
ing
reso
urce
s
By Ju
ne 2
000
All
dep
tsC
on
clus
ion
20:
dep
artm
ents
sho
uld
revi
ew,
by Ju
ne 2
000,
whe
ther
the
y ne
ed t
he
cap
abili
ty t
o ac
cess
and
use
larg
e G
over
nmen
t su
rvey
s su
ch a
s th
e FR
S an
d, if
so,
put
in
pla
ce t
he n
eces
sary
cap
acity
by
June
200
1.
Pote
ntia
l im
pac
t on
pay
bill
goin
g fo
rwar
d
Dep
ende
nt o
n
outc
ome
of r
evie
w
By e
nd 2
000
GES
/GSS
/
GO
RS/G
SR t
o re
view
join
tly
Co
ncl
usio
n 1
9: p
rofe
ssio
nal g
roup
s sh
ould
imm
edia
tely
rev
iew
the
imp
act
of d
evol
ved
pay
stru
ctur
es o
n re
tent
ion
and
inte
r-de
par
tmen
tal m
oves
and
con
side
r w
heth
er a
Gov
ernm
ent
wid
e in
crea
se in
pay
is r
equi
red
to e
nsur
e a
satis
fact
ory
sup
ply
of
high
qua
lity
anal
ysts
.
Cos
ts d
epen
dent
on
pre
cise
mod
aliti
es o
f
oper
atio
n
2001
Dep
tsC
on
clus
ion
18:
dep
artm
ents
sho
uld
pro
mot
e m
ore
vigo
rous
ly s
econ
dmen
ts t
o an
d ot
her
links
with
aca
dem
ia a
nd t
he p
rivat
e se
ctor
.
Smal
l inc
rem
enta
l cos
tIn
line
with
pro
gres
s on
Civ
il Se
rvic
e Re
form
Dep
tsC
on
clus
ion
17:
dep
artm
ents
sho
uld
intr
oduc
e m
ore
per
sona
l pro
mot
ion
to a
llow
indi
vidu
als
to c
hoos
e be
twee
n sp
ecia
list
and
gene
ralis
t ca
reer
pro
gres
sion
pat
hs.
Subs
titut
ion
for
exis
ting
recr
uitm
ent
pro
cedu
res
2001
Dep
tsC
on
clus
ion
16:
dep
artm
ents
sho
uld
seek
to
broa
den
the
poo
l fro
m w
hich
sp
ecia
lists
are
recr
uite
d to
incl
ude
fore
ign
natio
nal s
pec
ialis
ts w
here
ver
pos
sibl
e
Implementation strategy
81
Imp
act
in D
epts
Ad
dit
ion
alR
eso
urc
esR
equ
ired
By
Wh
enB
y W
ho
mC
on
clu
sio
n
6 m
onth
s of
G7
time
(£25
,000
)
June
200
0N
atio
nal S
tatis
ticia
n/
Stat
istic
s C
omm
issi
on
Co
ncl
usio
n 3
2:O
NS/
GSS
sho
uld,
as
par
t of
the
new
Nat
iona
l Sta
tistic
s p
lann
ing
pro
cess
,
revi
ew b
y Ju
ne 2
000
whe
ther
mor
e cr
oss-
dep
artm
enta
l dat
a co
llect
ion
shou
ld b
e
unde
rtak
en a
long
the
mod
el o
f th
e SE
U a
nd IG
CB.
3 m
onth
s of
G7
time
(£12
,500
)
Mar
ch 2
000
ON
SC
on
clus
ion
31:
the
Nat
iona
l Sta
tistic
ian
shou
ld e
nsur
e th
at r
egul
ar r
evie
w o
f ga
ps
is a
n
inte
gral
par
t of
the
new
Nat
iona
l Sta
tistic
s p
lann
ing
pro
cess
.
To b
e de
term
ined
by
busi
ness
cas
e
Part
of
CSR
rev
iew
ON
SC
on
clus
ion
30:
ON
S sh
ould
brin
g to
geth
er in
tere
sted
dep
artm
ents
by
June
200
0 to
ass
ess
whe
ther
the
re is
a b
usin
ess
case
for
a r
egul
ar s
urve
y of
indi
vidu
al w
ealth
and
ass
ets.
4 m
onth
s of
G7
time
(£17
,500
)
June
200
0O
NS
Co
ncl
usio
n 2
9:O
NS
shou
ld c
arry
out
a r
evie
w b
y Ju
ne 2
000
to e
stab
lish
whe
ther
the
y
shou
ld s
et u
p a
tea
m t
o co
-ord
inat
e th
e co
mp
ilatio
n of
loca
l are
a da
ta o
utsi
de t
he s
ocia
l
excl
usio
n ar
ena.
To f
orm
par
t of
CM
PS
fund
ing
June
200
0C
MPS
Co
ncl
usio
n 2
8: C
MPS
sho
uld
set
up a
wor
king
gro
up,
by Ju
ne 2
000,
to
exch
ange
exp
erie
nces
bet
wee
n de
par
tmen
ts a
nd e
xplo
re t
he f
utur
e ro
le o
f p
ilots
. Be
st p
ract
ice
guid
ance
sho
uld
set
out
the
exp
ecta
tion
that
dep
artm
ents
sho
uld
use
pilo
ts,
as p
art
of t
he
pol
icy
app
rais
al p
roce
ss,
whe
reve
r ap
pro
pria
te.
Non
eO
ngoi
ngG
SS u
sing
sp
ecia
list
netw
orks
Co
ncl
usio
n 2
7: G
SS s
houl
d di
ssem
inat
e be
st p
ract
ice
in t
he u
se o
f lo
ngitu
dina
l dat
a.
Part
of
Whi
te P
aper
imp
lem
enta
tion.
End
2000
ON
SC
on
clus
ion
26:
ON
S an
d le
ad d
epar
tmen
ts s
houl
d p
rom
ulga
te t
heir
stra
tegy
for
long
itudi
nal d
ata
to a
naly
sts
acro
ss G
over
nmen
t th
roug
h a
serie
s of
wor
ksho
ps
and
sem
inar
s.
Cos
t of
rev
iew
= 0
.5 o
f
G7
for
one
year
per
dep
artm
ent
June
200
0Re
leva
nt d
epts
Co
ncl
usio
n 2
5: d
epar
tmen
ts r
esp
onsi
ble
for
surv
eys
cove
ring
inco
me
shou
ld c
arry
out
are
view
by
end
June
200
0 of
the
cos
ts a
nd b
enefi
ts o
f m
ore
freq
uent
and
tim
ely
data
rele
ase.
Non
e re
qui
red
End
2000
ON
SC
on
clus
ion
24:
ON
S sh
ould
enc
oura
ge s
pec
ialis
ts t
o be
mor
e cr
eativ
e in
the
ir us
e of
dat
a
to h
elp
fill
spec
ific
data
gap
s w
ithin
sur
veys
. Th
roug
h se
min
ars
and
wor
ksho
ps
exam
ple
s of
best
pra
ctic
e co
uld
be e
xpos
ed t
o a
wid
er a
udie
nce.
Addi
ng It
Up
82
Imp
act
in D
epts
Ad
dit
ion
alR
eso
urc
esR
equ
ired
By
Wh
enB
y W
ho
mC
on
clu
sio
n
Part
of
conc
lusi
on 4
Ann
ually
fro
m 2
001
HM
T (C
ORE
)C
on
clus
ion
38:
the
revi
ew o
f an
alys
is t
o be
con
duct
ed b
y th
e C
ORE
tea
m in
HM
Tre
asur
y
(see
con
clus
ion
4) s
houl
d ad
dres
s ex
plic
itly
the
need
for
and
pro
visi
on o
f jo
intly
dev
elop
ed
mic
roec
onom
ic m
odel
s.
Ong
oing
adm
inis
trat
ive
cost
of
iden
tifyi
ng a
nd
invi
ting
app
rop
riate
indi
vidu
als
Mar
200
0H
MT
Co
ncl
usio
n 3
7:a
mod
ellin
g ne
twor
k ch
aire
d by
the
Chi
ef E
cono
mis
t in
the
Tre
asur
y Pu
blic
Serv
ice
Dire
ctor
ate
shou
ld b
e es
tabl
ishe
d at
tec
hnic
al le
vel b
y sp
ecia
lists
with
in
Gov
ernm
ent
to p
rom
ote
exch
ange
of
good
pra
ctic
e an
d to
sig
nal t
he im
por
tanc
e of
mod
ellin
g w
ithin
Gov
ernm
ent
the
by M
arch
200
0.
Cos
t lik
ely
to b
e
equi
vale
nt t
o th
e
esta
blis
hmen
t of
a n
ew
rese
arch
cen
tre
on
aver
age
arou
nd
£500
,000
ann
ually
End
2000
Chi
ef E
cono
mis
t in
the
Trea
sury
Pub
lic S
ervi
ces
Dire
ctor
ate
Co
ncl
usio
n 3
6:th
e C
hief
Eco
nom
ist
in t
he T
reas
ury
Publ
ic S
ervi
ces
Dire
ctor
ate
shou
ld
enco
urag
e th
e ES
RC t
o ad
opt
the
aim
of
mak
ing
the
UK
a ce
ntre
of
inte
rnat
iona
l
exce
llenc
e in
ap
plie
d m
icro
econ
omic
ana
lysi
s by
200
5.
No
cost
abo
ve t
he
cost
s of
mod
el
deve
lop
men
t
With
imm
edia
te e
ffect
All
dep
tsC
on
clus
ion
35:
outs
ide
exp
ertis
e sh
ould
be
invo
lved
in m
odel
dev
elop
men
t fr
om t
he
outs
et t
hrou
gh a
n ex
tern
al s
teer
ing
grou
p o
r ot
her
mec
hani
sms.
Co-
oper
atio
n an
d
liais
on w
ith m
odel
ling
pan
el.
Prec
ise
imp
act
dep
ende
nt o
n te
rms
of
refe
renc
e fo
r m
odel
revi
ew s
et b
y p
anel
.
Dire
ct c
ost:
£20
000
(5ou
tsid
ers;
1 d
ay,
4tim
es a
yea
r)
Rese
arch
bud
get
(for
audi
t) =
£15
0,00
0
– de
pen
dent
on
wor
k
pro
gram
me
June
200
0Im
ple
men
tatio
n gr
oup
and
dep
ts
Co
ncl
usio
n 3
4:a
Mod
ellin
g A
cade
mic
Pan
el s
houl
d be
est
ablis
hed
to d
efine
and
sup
ervi
se
the
pro
cess
of
pub
licat
ion
and
exte
rnal
rev
iew
of
long
ter
m m
odel
s us
ed in
Gov
ernm
ent.
This
Pan
el s
houl
d be
cha
ired
by t
he C
hief
Eco
nom
ist
in t
he T
reas
ury
Publ
ic S
ervi
ces
Dire
ctor
ate.
It s
houl
d in
clud
e bo
th c
ivil
serv
ants
and
aca
dem
ics.
Part
of
gene
ral b
usin
ess
pla
nnin
g
No
addi
tiona
l cos
ts
abov
e th
ose
in t
he
busi
ness
pla
nnin
g
pro
cess
Part
of
defin
ition
of
anal
ysis
in s
upp
ort
of
PSA
tar
gets
All
Dep
tsC
on
clus
ion
33:
dep
artm
ents
sho
uld
revi
ew t
heir
mod
ellin
g ca
pac
ity f
rom
firs
t p
rinci
ple
s to
mak
e su
re t
hat
thei
r m
odel
ling
wor
k re
mai
ns a
pp
rop
riate
to
the
curr
ent
rang
e of
dep
artm
enta
l obj
ectiv
es.
Implementation strategy
83
Imp
act
in D
epts
Ad
dit
ion
alR
eso
urc
esR
equ
ired
By
Wh
enB
y W
ho
mC
on
clu
sio
n
Cha
rgin
g im
plic
atio
ns
to b
e co
nsid
ered
as
par
t of
the
rev
iew
pro
cess
End
2000
All
dep
tsC
on
clus
ion
41:
dep
artm
ents
sho
uld
exp
lore
with
the
priv
ate
sect
or t
he f
easi
bilit
y of
acce
ssin
g an
d us
ing
priv
ate
sect
or d
ata
sour
ces.
No
addi
tiona
l res
ourc
esTo
fit
in w
ith c
ivil
serv
ice
refo
rm a
gend
a
CO
Co
ncl
usio
n 4
0:th
e C
ivil
Serv
ice
refo
rm p
lans
to
iden
tify
100
key
task
s fo
r p
rivat
e se
ctor
seco
ndee
s sh
ould
incl
ude
a nu
mbe
r of
pla
ces
for
inte
rnat
iona
lly r
ecog
nise
d sp
ecia
lists
.
Up
to
£50,
000
Jun
2000
All
dep
tsC
on
clus
ion
39:
a ‘c
halle
nge
fund
’ of
£50,
000
(adm
inis
tere
d by
the
CO
RE t
eam
in t
he
Trea
sury
) sh
ould
be
esta
blis
hed
to f
und
five
acad
emic
sum
mer
pla
cem
ents
in W
hite
hall
each
yea
r st
artin
g fr
om Ju
ly 2
000.
Addi
ng It
Up
84
A1.1 The creation of the Performance andInnovation Unit (PIU) was announced by thePrime Minister on 28 July 1998 as part of thechanges following a review by Sir RichardWilson of the effectiveness of the centre ofgovernment. The PIU’s aim is to improve thecapacity of government to address strategic,cross-cutting issues and promote innovationin the development of policy and in thedelivery of the government’s objectives.The PIU is part of the drive for better, morejoined-up government. It acts as a resourcefor the whole of government, tackling issuesthat cross public sector institutionalboundaries on a project basis.
A1.2 The Unit reports direct to the PrimeMinister through Sir Richard Wilson and isheaded by a Senior Civil Servant, Mr SumaChakrabarti. It has a small central team thathelps recommend project subjects, managesthe Unit’s work and follows up projects’recommendations with departments. Workon the projects themselves is carried out bysmall teams assembled both from insideand outside government. About half ofthe current project team staff are drawnfrom outside Whitehall, including fromprivate sector consultancies, academiaand local government.
A1.3 The first set of PIU projects wasannounced by the Prime Minister inDecember 1998. These were:
• Developing Electronic Commerce inthe UK – how to make the UK the world’sbest environment for electronic commerce,ensuring that the UK benefits fully fromthe single fastest growing marketplacein the global economy – publishedSeptember 1999;
• a small working group separatelyconsidered the issue of Encryption andLaw Enforcement. The report, publishedin May 1999, sets out the issuessurrounding encryption, e-commerce andlaw enforcement, and recommendationsto achieve better balanced governmentpolicy in this area;
• Objectives for Rural Economies –examining the differing needs of local ruraleconomies, and the key factors affectingperformance, so as to establish clearobjectives for government policiesinfluencing the future developmentof rural economies – publishedDecember 1999;
• Active Ageing – how to improve thewell-being and quality of life of olderpeople by helping them to remain active.The study will identify ways of increasingthe employment opportunities for olderpeople, by examining the incentives forbusinesses to employ and retain olderpeople and for individuals to remain inpaid or voluntary work (due to reportearly 2000);
ANNEX A1. THE ROLE OF THE PERFORMANCE ANDINNOVATION UNIT
The Role
of the
Performance and
Innovation Unit
85
• Central Government’s Role atRegional & Local Level – getting theright institutional arrangements andrelationships in place for joined-up deliveryof central government policies in regionsand communities (due to report by early2000); and
• Accountability and Incentives forJoined-Up Government – examininghow current accountability arrangementsand incentive systems can be reformed tofacilitate joined-up policy making anddelivery, for example by promotingachievement of joint objectives whichrequire co-operation between departments(due to report early 2000).
A1.4 The Unit is also separately identifyingthe key long-term Strategic Challengesthat government will have to face, as referredto in the government’s ModernisingGovernment White Paper, published in April1999. This work will help departments andother organisations to look beyond theirexisting policies towards the government’slong-term goals.
Further projectsA1.5 A further set of projects has since beenannounced:
• The Post Office Network – to developa strategic view of the Post Office networkfollowing the Post Office White Paper,including picking up the commitmentin the White Paper to develop minimumcriteria access to Post Office services;
• Trade and Social, Health andEnvironmental Objectives on theGlobal Stage – to identify a coherent setof principles for handling social, health andenvironmental issues in international fora,with a particular focus on the WTO buttaking also account of other internationalinstitutions such as United Nationsagencies;
• The Pursuit and Seizure of CriminalAssets – to evaluate the effectiveness offollowing the money trail and seizingcriminal assets as a technique in fightingserious and organised crime and, in thelight of the analysis, develop a newstrategy to exploit this approach morevigorously; and
• Strategic Options for the ElectronicDelivery of Government Services –this study will, in the light ofdevelopments in technology, take astrategic view of which public servicesshould be delivered by electronic means,and look at the options for securingdelivery of these services, includingthe respective roles of the public andprivate sectors.
Steering GroupGus O’Donnell – Director of MacroeconomicPolicy and Prospects (HMT) and Head of theGovernment Economic Service. Chair of theSteering Group
Norman Glass – Deputy Director, PublicServices Directorate, HMT
Melanie Dawes – Work Incentives and PolicyAnalysis Team, HMT
Sharon White – No 10 Policy Unit
Andrew Dilnot – Director, Institute for FiscalStudies
Alan Woods – Policy Director, Department forSocial Security
Dennis Roberts – Group Director for Financeand Corporate Services, Office for NationalStatistics
Andrew Burchell – Director of EnvironmentalProtection Strategy, DETR
Paul Gregg – Economic Adviser, HMT
John Vickers – Director and Chief Economist,Bank of England
Paul Johnson – Financial Services Authority
Professor Ron Amann – Centre for Managementand Policy Studies, Cabinet Office
Penelope Rowlatt – Director, EuropeanEconomic Research
Stephen Aldridge – Senior Economic Advisor,Performance and Innovation Unit
Project TeamWilliam Jordan – Team leader, Performanceand Innovation Unit
Jonathan Ockenden – Team member,Performance and Innovation Unit
Najma Rajah – Team member, Performanceand Innovation Unit
The project team are also grateful foradditional help from Stephen Aldridgeand Stephen Gifford from the PIU CentralEconomics team and Dennis Robertsfrom ONS.
Project SponsorSir Richard Wilson, Head of the Home CivilService
ANNEX A2. MEMBERS OF THE PROJECT STEERINGGROUP AND TEAM
Addi
ng It
Up
86
A3.1 This Annex sets out the main conclusionsfrom the individual case studies and drawssome overall conclusions. The full text of thecase studies is published on the PIU website(www.cabinet-office.gov.uk/innovation).
A3.2 The case studies fall into twomain types:
• modelling case studies: these lookclosely at particular models as part ofthe policy making process. They considerthe technical strengths and limitationsof the models as well as the institutionaland policy setting in which the modelsare used;
• policy case studies: these examine morewidely the extent to which analysis hasunderpinned past policy formulation.
Case study findingsA3.3 A thumbnail sketch our findings fromthe modelling case studies suggests that:
• HMT and DSS tax/benefit modellingis proficient within its limitations.Tax/benefit modelling takes place in acompetitive environment and the Treasurymodel has recently been audited by IFS.However, neither model can model allbehavioural impacts of policy (there aretechnical difficulties to overcome) or take-up (the DSS model is calibrated againsttake-up through an off-model adjustment);
• the DETR household formationmodel is a statistical model that workson the basis of demographic data andsophisticated trend extrapolationtechniques. In the past, it has been usedto set the agenda for land-use planning,but is now only advisory. It would be morehelpful as a policy tool if it took accountof economic variables. There are plansto develop the model in this direction;
• the Home Office property crimemodels explain the level of propertycrime in terms of demographic and macro-economic factors. They do notinclude variables to reflect policy becauseof the difficulties of obtaining a time seriesof these type of variables. The modelsare useful in providing a baseline againstwhich the impacts of policies can bemeasured. But a micro-model whichcaptured the response of individuals’behaviour to economic, policy and otherchanges would be more helpful as a toolto help assess the impact of policy options.Home Office are drawing in externalacademic advice to help incorporate otherrelevant factors into the analysis. Theirapproach will therefore aim to take intoaccount policy impacts in appraisal andevaluation frameworks, to which theircrime modelling is only one input;
• the DETR National Road TrafficForecasting Model is a statisticalrepresentation of the capacity of the roadsystem. It is based on extensive data andwith a sound technical basis, takes accountof behavioural responses to supply sideconstraints (i.e. the clogging up of the
ANNEX A3. MAIN FINDINGS FROM THE CASE STUDIES
Main Findings
from the
Case Studies
87
road network). It helps policy makersunderstand the potential impact of a widerange of options. A work programme isin place to develop the model further.In particular, there are plans to build a“geographical” model to complement theexisting “statistical” model (i.e. to find out,amongst other things, where capacity isbeing fully used);
• LCD modelling of civil legal aidexpenditure gives an indicative ratherthan precise estimates of future spendingon legal aid. But the model is sufficientfor departmental needs, especially giventhat the present system is shortly to bechanged. The present model comparesfavourably in terms of accuracy with itsimmediate predecessor;
• GAD modelling of population issophisticated. Details of the model arepublic and open to external scrutiny. UKmodelling compares favourably with thatof other countries. But outputs areuncertain: there is no way to tell whethera country is in the midst of a baby boomor has a rising trend in fertility or forhow long improvements in mortalitywill continue. Forward projections onmigration rates are particularly uncertain;
• OFWAT efficiency models aresophisticated, externally audited, anddesigned to be capable of withstandingscrutiny by the Competition Commission.Off-model adjustments are made to modeloutputs in setting efficiency targets forwater companies;
• DETR’s model of restructuring socialrents works well within strictly definedparameters. But the lack of small area dataand of behavioural modelling means thatthe real-world impacts of policy changeare unmodelled – and unknown;
• NHS waiting list modelling is basedon a straightforward spreadsheet. Themodel itself is sound. But it is limited byuncertainties over inputs and the behaviourunderlying the basic data. Work is beingundertaken to reduce these uncertainties.The first results of a literature survey onthe likely behavioural response of GPs tochanges in waiting list numbers are duewith the department in early 2000;
• DSS PENSIM modelling of pensionerincomes took 7 years to get operationalafter external contractors delivered amodel whose workings were unclear todepartmental analysts. The model is basedon survey data now 10 years out of date,does not model all sources of pensionerincome and has difficulties in producingsensitivity analysis. DSS has plans toimprove the model on a rolling basis andto replace it with a revised version,PENSIM II;
• PSSRU’s long-term care modelling forDH is technically sound, but is much lessadvanced than comparable US modellingdue to a shortage of data. For example,the UK has a lack of longitudinal dataon ageing and there is no data on howdisability benefits affects spending onlong-term care;
• Customs and Excise analysis ofrevenue from tobacco duty was basedaround a reliance on a model which hadgiven very accurate revenue predictions ina closed market but was not well suited tothe open borders introduced by the SingleEuropean Act. The model has now beenadapted to allow explicitly for cross-bordershopping and smuggling.
A3.4 The policy case studies providea fuller picture of the role that analysisand modelling play in the formulation ofgovernment policy. The policy case studiessuggest the following conclusions:
Addi
ng It
Up
88
• the DfEE labour market policy casestudy highlights the relationship betweeneconomic theory, the analysis carried outin Government departments and the dataon which that analysis rests. It also showshow a change in policy arising from aMinisterial initiative was subsequentlyexplained by analysis of how the labourmarket works;
• the DSS Green paper on pensionshighlights the difficulties in formulatingpolicies in areas that are complex andrelate to the long term. In particular, thiscase study demonstrates the strengths andlimitations of PENSIM and the impact onthe policy process of the PensionsProvision Group, which comprised externalexperts whose role was to gather togetherbackground information to supportthe reforms;
• DETR analytical support to policydevelopment on air quality highlightsa comparative success. Policy is closelyinformed by a team of bedded-outscientists working in the policy line and bya DETR-led interdepartmental group oncosts and benefits. The Group’s work wasin time to influence EU targets but not theinitial development of provisional domestictargets. Further work is required onmonetisation of health benefits and onthe impacts of industrial emissions onair quality;
• FCO and DCMS PSA targets were notfully backed by analysis in the 1997-98Comprehensive Spending Review. Theposition should improve in the presentSpending Review;
• Cabinet Office CITU targets on theelectronic delivery of government serviceswere based on benchmarking againstthe private sector and internationalcomparisons. They were not underpinnedby any detailed dynamic analysis ofachievability. Nor at the time of the initial
case study work had there been anyappraisal of which services can bedelivered most effectively, of the costs andbenefits of electronic delivery at the levelof “whole of government” and whichclients wish to receive in this way.This work has, however, since beencommissioned.
Overall conclusionsA3.5 In relation to long term models thecase studies suggest:
• models are not a panacea. Even wherethe technical standard of modelling is veryhigh, models are limited in theircontribution to important policy questions.For example, the OFWAT models ofefficiency cannot explain the role ofcompany-specific factors in determiningwater company expenditure levels;
• some behavioural effects areunmodelled. This applies, for example,to the LCD model of legal aid certificates(with respect to the behaviour ofsolicitors), the DSS model of housingbenefit, the DETR model of social rentsand both tax and benefit models;
• external scrutiny does play a rolein model development, althoughno department has systematicprocedures for ensuring this occurs.GAD, DSS, in particular, the Home Officeand OFWAT have exposed their models ina variety of fora;
• evaluation of model outputs is rarelyundertaken. This is in contrast to themodelling of the macro economy. Theexception to this is the context of the GADmodel where the model predictions arepublished and compared with actualoutcomes on a biennual basis.
Main Findings
from the
Case Studies
89
A3.6 On model building the case studiessuggest that:
• there appears to be no systemicpattern in the use of particular kindsof specialists for building models.The LCD model was developed byoperational researchers; the GAD model bystatisticians and actuaries; the OFWAT andDETR social housing model by economists;and the PENSIM model by outsideconsultants. In the case of the LCD model,it is not clear that either statisticians oreconomists could have contributed much.Conversely, there may be a role foreconomists in the development of theGAD model;
• the evidence on contracting-outmodel building is mixed. The PENSIMcase study reveals the problems that canarise from contracting out the buildingof models to external consultants.On the other hand, OFWAT have hada wholly successful liaison withacademic consultants.
A3.7 On data issues:
• many different types of data sourcesare drawn on in the course ofanalysis and modelling. Models are notonly based on household surveys (IGOTM,PSM and the DETR social rents model) butalso administrative data (LCD models oflegal aid and Home Office property crimemodel). However, there appear to beno instances where attempts have beenmade to link up different data sources.Longitudinal data was drawn on onlyin the case of long-term care;
• in nearly all cases data availabilitylimited the range of questions thatanalysis and models could usefullyanswer.
A3.8 On support for the policy formulationprocess:
• there was not much evidence themodels were used in a wider contextto enhance understanding. There werea number of examples where models werelargely viewed as tools which producednumbers. e.g. PENSIM and PSM. Thereappeared to be little evidence of use ofmodels to increase understanding aboutthe underlying factors influencing policy;
• the use of models to explore theimpacts of policies has beenrelatively limited. This applies to theGAD population projections model.The HO office model includes no policyvariables, largely reflecting data limitations.The C&E revenue model also used not toinclude policy variables as the impact ofthe increase in tobacco duty on smugglingwas not fully captured. Steps have nowbeen taken to correct this;
• there is a mixed picture about theextent to which policy customersunderstand the limitations of theoutputs of analysis and modelling.In some cases the limits of analysis are wellunderstood. This is the case for the costsand benefits of meeting particular airquality targets and the outputs of the DHwaiting list and waiting times models. Inother cases, policy customers may be lessclear about the limitations of models. Tosome extent there is some correlationbetween policy customers’ understandingand the location of analysts.
Addi
ng It
Up
90
Main Findings
from the
Case Studies
91
Met
hodo
logi
cal i
ssue
s
incl
udin
g ar
bitr
ary
mor
talit
y as
sum
ptio
ns
tobe
imp
rove
d. M
ore
tran
spar
ent
off-
mod
el
pro
cess
mig
ht b
e
adop
ted
to s
how
how
e.g.
med
ical
imp
rove
men
ts a
re
fact
ored
into
cha
nges
inth
e m
orta
lity
rate
s.
Prov
ides
con
sist
ent
num
bers
acr
oss
govt
.
Inte
rnat
iona
lly w
ell
rega
rded
. G
ood
use
of
sens
itivi
ty a
naly
sis.
Mig
ratio
n is
a p
robl
em
area
for
all
pop
ulat
ion
mod
els,
but
the
ap
pro
ach
is r
elat
ivel
y so
phi
stic
ated
in c
omp
aris
on w
ith o
ther
EU c
ount
ries.
No
abili
ty t
o fo
reca
st
turn
ing
poi
nts.
Con
sist
ent
unde
r-
pre
dict
ion.
Met
hodo
logy
is d
iscu
ssed
at E
U le
vel t
hrou
gh
Euro
stat
.
Out
put
s us
ed b
y D
ETR,
HO
, D
SS,
DH
, O
NS.
Cen
sus,
sta
tistic
s fr
om
Regi
stra
tion
of B
irths
Mar
riage
s an
d D
eath
s.
Dem
ogra
phi
c
pro
ject
ion
mod
el.
Pop
ulat
ion
pro
ject
ions
.
Poss
ible
mer
ger
with
HM
T ta
x be
nefit
mod
el.
Imp
rove
d m
odel
ling
of
beha
viou
ral r
elat
ions
hip
s,
dem
ogra
phi
c an
d
econ
omic
cha
nges
betw
een
surv
eys
and
low
inco
me
hous
ehol
ds.
Two
stag
e an
alys
is w
ould
be
app
rop
riate
: th
e fir
st
stag
e w
ould
sho
w fi
rst
roun
d im
pac
ts b
ased
on
the
mod
el o
utp
ut a
nd
the
seco
nd w
ould
show
the
off-
mod
el
imp
acts
of
pot
entia
l
beha
viou
ral e
ffect
s.
Use
r fr
iend
ly w
ith s
ome
usef
ul b
asic
dyn
amic
effe
cts
(iden
tifica
tion
offlo
ater
s).
Mod
ellin
g of
tak
e-up
(not
entit
lem
ent)
is
crud
e.
Lack
of
beha
viou
ral
mod
ellin
g.
Mod
els
outp
ut u
sed
byot
her
dep
ts
(HM
Tan
dIR
).
Som
e co
mp
aris
on w
ith
othe
r m
odel
s in
the
pas
t.
Mor
e p
lann
ed f
or t
he
futu
re.
Fam
ily R
esou
rces
Sur
vey
good
siz
e sa
mp
le,
but
suffe
rs u
nder
sam
plin
g at
the
upp
er e
nd o
f th
e
inco
me
dist
ribut
ion.
Ther
e ar
e al
so s
igni
fican
t
time
lags
.
Stat
ic
mic
rosi
mul
atio
n
mod
el.
Tax
and
bene
fit
mod
ellin
g (D
SS).
Sug
ges
tio
ns
for
imp
rove
men
tSt
ren
gth
sLi
mit
atio
ns
Exte
rnal
in
pu
tD
ata
sou
rces
and
qu
alit
yTy
pe
of
An
alys
isP
oli
cy a
rea
Addi
ng It
Up
92
New
ana
lysi
s –
data
colle
ctio
n an
d m
odel
ling
wor
k –
with
a v
iew
to
imp
rovi
ng u
nder
stan
ding
and
assi
stin
g co
mba
t of
smug
glin
g ha
s be
en
unde
rtak
en.
Will
nee
d to
be r
eass
esse
d in
the
ligh
t
of n
ew e
vide
nce.
Att
emp
ts t
o re
spon
d to
the
deve
lop
ing
shor
tfal
l
have
sho
wn
good
use
of
data
and
com
bini
ng
spec
ialis
t kn
owle
dge.
Ana
lysi
s, b
ased
on
a
mod
el e
stim
ated
fro
m
data
prio
r to
the
intr
oduc
tion
of t
he s
ingl
e
mar
ket
in 1
993
has
beco
me
pro
gres
sive
ly le
ss
relia
ble
with
the
gro
wth
of s
mug
glin
g.
Con
sulta
tion
with
Esse
xU
nive
rsity
on
pric
eel
astic
ities
.
Dat
a ga
ther
ed f
rom
:
Inte
rnat
iona
l Pas
seng
er
Surv
ey;
Air
Pass
enge
r
Surv
ey;
Vario
us s
urve
y
base
d m
etho
ds o
f
asse
ssin
g sm
uggl
ing
and
GH
S.
All
data
sub
ject
to
pro
blem
s of
rec
ordi
ng
illic
it an
d ill
egal
activ
ityan
d p
robl
em
ofsu
rvey
dat
a.
Econ
omet
ric
mod
ellin
g of
exp
ecte
d re
venu
e
and
subs
eque
nt
anal
ysis
of
shor
tcom
ings
of
that
mod
el.
Als
o us
es s
urve
y to
cap
ture
sm
uggl
ing.
Ana
lysi
s of
imp
act
of
smug
glin
g on
toba
cco
reve
nues
.
Shou
ld lo
ok m
ore
clos
ely
at in
tegr
atin
g m
icro
and
mac
ro e
cono
mic
fac
tors
.
Prov
ides
som
e
unde
rsta
ndin
g of
the
exog
enou
s
mac
roec
onom
ic
pre
ssur
es o
n cr
imin
al
beha
viou
r.
Mod
els
do n
ot c
onta
in
pol
icy
varia
bles
nor
do
they
pro
vide
muc
h
unde
rsta
ndin
g in
divi
dual
driv
ers
of c
rime.
Poss
ible
dec
line
in
pre
dict
ive
pow
er o
f th
e
mod
el.
Hom
e O
ffice
reco
gnis
e th
at p
olic
y
imp
acts
can
onl
y p
rop
erly
be t
aken
into
acc
ount
in
app
rais
al/e
valu
atio
n
fram
ewor
k.
Peer
rev
iew
by
acad
emic
s
and
econ
omet
ricia
ns.
Publ
ishe
d N
ovem
ber
1999
for
ext
erna
l scr
utin
y
and
revi
ew.
Sem
inar
s
from
199
9 to
dra
w in
exte
rnal
exp
ertis
e an
d
inco
rpor
ate
othe
r fa
ctor
s.
Reco
rded
crim
e st
atis
tics
whi
ch a
re w
idel
y
rega
rded
as
a p
oore
r
serie
s th
an t
he B
ritis
h
Crim
e Su
rvey
, bu
t
whi
chp
rovi
ded
a
long
erda
ta s
ourc
e.
Relia
ble
dem
ogra
phi
c
and
mac
roec
onom
ic
stat
istic
s.
Two
time
serie
s
econ
omet
ric m
odel
s.
Exp
lain
ing
pro
per
ty
crim
e.
Sug
ges
tio
ns
for
imp
rove
men
tSt
ren
gth
sLi
mit
atio
ns
Exte
rnal
in
pu
tD
ata
sou
rces
and
qu
alit
yTy
pe
of
An
alys
isP
oli
cy a
rea
Main Findings
from the
Case Studies
93
Furt
her
evid
ence
is b
eing
gath
ered
to
refin
e th
e
anal
ysis
fur
ther
.
A g
ood
new
s st
ory.
Diff
eren
t p
arts
of
the
anal
ytic
com
mun
ity
com
bine
d w
ell.
The
anal
ysis
was
influ
entia
l at
the
EU le
vel.
Dep
artm
ents
wor
ked
reas
onab
ly w
ell t
oget
her.
Prec
isio
n of
est
imat
es
limite
d by
dat
a
cons
ider
atio
ns.
Wea
knes
ses
in im
pac
ts o
n
indu
stry
refl
ectin
g so
me
(inev
itabl
e) d
iffer
ence
s of
dep
artm
enta
l vie
w.
Exte
rnal
res
earc
h
com
mis
sion
ed f
rom
SPR
U
and
ETSU
.
Initi
al fi
ndin
gs w
ent
to
cons
ulta
tion
with
pro
duct
ive
feed
back
gain
ed f
rom
var
ious
lobb
y gr
oup
s.
Dat
a on
em
issi
ons
colle
cted
thr
ough
exte
nsiv
e m
onito
ring
pro
gram
me.
Dat
a on
hea
lth e
ffect
s
less
firm
, es
pec
ially
at
the
loca
l lev
el.
Com
bina
tion
of
scie
ntifi
c, e
cono
mic
and
engi
neer
ing
inp
uts
to a
sses
s co
sts
and
bene
fits
of
imp
rovi
ng a
ir
qua
lity.
Nat
iona
l Air
Qua
lity
Stra
tegy
.
Rese
arch
pro
gram
me
for
stre
ngth
enin
g q
ualit
y of
mod
el in
put
s in
pla
ce.
Con
sist
ent
fram
ewor
k.
Effe
ctiv
e ne
gotia
ting
tool
.
Mod
el h
eavi
ly d
epen
dent
on t
he q
ualit
y of
its
inp
uts.
Lack
of
seco
nd r
ound
and
beha
viou
ral e
ffect
s
inth
e m
odel
.
Lim
ited
rang
e of
use
s fo
r
pol
icy
eval
uatio
n.
No
form
al s
crut
iny,
but
the
mod
el h
as b
een
pre
sent
ed a
t co
nfer
ence
s
and
oper
ates
in
com
pet
ition
with
priv
ate
sect
or m
odel
s.
Adm
inis
trat
ive
data
on
adm
issi
ons
and
wai
ting
times
. M
ore
com
ple
te
data
in a
num
ber
of a
reas
wou
ld a
llow
mor
e p
reci
se
mod
el t
o be
dev
elop
ed.
Spre
adsh
eet
mod
el.
NH
S Tr
eatm
ent
Wai
ting
Tim
es.
Sug
ges
tio
ns
for
imp
rove
men
tSt
ren
gth
sLi
mit
atio
ns
Exte
rnal
in
pu
tD
ata
sou
rces
and
qu
alit
yTy
pe
of
An
alys
isP
oli
cy a
rea
Addi
ng It
Up
94
The
basi
s on
whi
ch L
egal
Aid
pay
men
ts a
re m
ade
is
goin
g to
cha
nge
soon
(with
a s
hift
to
bloc
k
cont
ract
ing)
. In
vie
w o
f
this
fur
ther
dev
elop
men
t
of t
he m
odel
wou
ld n
ot
be a
pp
rop
riate
.
The
mod
el p
rodu
ces
rela
tivel
y ac
cura
te s
hort
term
for
ecas
ts.
It c
an a
lso
be u
sed
to h
elp
incr
ease
unde
rsta
ndin
g of
pot
entia
l beh
avio
ural
imp
acts
suc
h as
law
yers
resp
onse
s to
cha
nges
inp
olic
y.
The
mod
el d
oes
not
accu
rate
ly f
orec
ast
indi
vidu
al p
rogr
amm
e
spen
ding
, ca
nnot
mod
el
pat
tern
s of
issu
ing
lega
l
aid
cert
ifica
tes
and
does
not
cap
ture
reg
iona
l
varia
tion.
Num
ber
of p
olic
y
que
stio
ns w
hich
can
be
answ
ered
are
cons
eque
ntly
lim
ited.
The
mod
el is
not
exp
osed
to e
xter
nal s
crut
iny.
The
outp
ut o
f th
e m
odel
is
chec
ked
agai
nst
that
of
the
Lega
l Aid
Boa
rd’s
less
sop
hist
icat
ed m
odel
.
The
mod
el is
bas
ed o
n
adm
inis
trat
ive
data
fro
m
the
Lega
l Aid
Boa
rd.
Unt
il
rece
ntly
thi
s w
as o
f a
high
qua
lity,
but
the
intr
oduc
tion
of a
new
com
put
er s
yste
m a
pp
ears
to h
ave
redu
ced
the
data
qua
lity.
Stat
istic
al p
roje
ctio
n
mod
el.
Lega
l Aid
exp
endi
ture
.
The
dep
artm
ent
has
a
wel
l-dev
elop
ed f
orw
ard
pla
n fo
r fu
ture
deve
lop
men
t co
verin
g
bett
er m
odel
ling
of
dem
and
grow
th,
trip
leng
th,
the
link
betw
een
vehi
cle
fuel
effi
cien
cy a
nd
fuel
pric
es,
spee
d-flo
w
assu
mp
tions
, lin
king
in
urba
n p
olic
y m
odel
s,
and
imp
rovi
ng n
atio
nal
netw
ork
data
in t
he
base
year
.
The
mod
el is
uni
que
in
that
no
othe
r co
untr
y ha
s
such
a w
ell-d
evel
oped
mod
el c
apab
le o
f
answ
erin
g a
wid
e
rang
eof
pol
icy
que
stio
ns,
for
exam
ple
cong
estio
n ta
xes.
The
mod
el o
f ro
ad
cap
acity
is o
nly
a
stat
istic
al r
epre
sent
atio
n.
The
mod
el o
ffers
no
insi
ghts
into
wha
t ca
uses
switc
hing
bet
wee
n
alte
rnat
ive
mod
es o
f
tran
spor
t; h
ence
it d
oes
not
offe
r an
swer
s to
allt
he in
tere
stin
g
pol
icy
que
stio
ns.
The
mod
el w
as
deve
lop
ed w
ith t
he in
put
of a
ste
erin
g gr
oup
of
acad
emic
s an
d tr
ansp
ort
pro
fess
iona
ls.
NTS
and
Cen
sus
data
–
over
all t
he q
ualit
y of
the
data
is h
igh
and
bett
er
than
tha
t fo
und
in
othe
rco
untr
ies.
Stat
istic
al a
nd
econ
omet
ric m
odel
.
Nat
iona
l roa
d
tran
spor
t fo
reca
sts.
Sug
ges
tio
ns
for
imp
rove
men
tSt
ren
gth
sLi
mit
atio
ns
Exte
rnal
in
pu
tD
ata
sou
rces
and
qu
alit
yTy
pe
of
An
alys
isP
oli
cy a
rea
Main Findings
from the
Case Studies
95
The
deve
lop
men
t of
the
seco
nd s
et o
f PS
A t
arge
ts
shou
ld b
e a
mor
e fa
mili
ar
pro
cess
and
con
seq
uent
ly
bett
er.
Furt
her
thou
ght
will
nee
d to
be
give
n,
how
ever
, to
the
pro
blem
of e
stab
lishi
ng b
asel
ines
.
The
pro
cess
of
sett
ing
PSA
tar
gets
for
the
firs
t
time
usef
ully
rev
eale
d
man
y of
the
hith
erto
unid
entifi
ed p
robl
ems.
Ther
e w
ere
a nu
mbe
r of
pro
blem
s su
rrou
ndin
g
the
anal
ysis
und
erp
inni
ng
PSA
tar
gets
incl
udin
g th
e
tight
tim
esca
le,
the
evol
utio
n fr
om O
PAs
to
PSA
tar
gets
cre
ated
diffi
culti
es a
s di
d
the
unfa
mili
ar a
nd
evol
ving
pro
cess
.
Dev
elop
ed in
tan
dem
with
HM
T.
Not
rel
evan
t.A
naly
sis
of
dep
artm
enta
l dat
a.
Form
ulat
ion
of P
SA
targ
ets.
Ther
e ar
e a
num
ber
of
area
s fo
r fu
rthe
r
deve
lop
men
t –
anal
ysis
of
cost
s an
d be
nefit
s of
over
all p
rogr
amm
e an
d
spec
ific
elem
ents
with
in
it, m
odel
ling
the
futu
re
cour
se o
f te
chno
logi
cal
pro
gres
s an
d ap
pra
isal
of
diffe
rent
rou
tes
for
achi
evin
g ta
rget
s.
The
anal
ysis
mad
e us
e of
over
seas
evi
denc
e.
Lim
ited
anal
ytic
al in
put
into
the
set
ting
of t
he
targ
et.
CIT
U h
as n
o
econ
omis
ts o
r op
erat
iona
l
rese
arch
ers.
Non
e in
the
pas
t,
alth
ough
a g
roup
of
Info
rmat
ion
Age
Gov
ernm
ent
Cha
mp
ions
has
been
est
ablis
hed
chai
red
by E
-env
oy
Ale
xA
llan.
Not
rel
evan
t.C
omp
arat
or
info
rmat
ion
from
priv
ate
sect
or
and
over
seas
.
Sett
ing
of t
arge
ts f
or
elec
tron
ic d
eliv
ery
of
gove
rnm
ent
serv
ices
.
Sug
ges
tio
ns
for
imp
rove
men
tSt
ren
gth
sLi
mit
atio
ns
Exte
rnal
in
pu
tD
ata
sou
rces
and
qu
alit
yTy
pe
of
An
alys
isP
oli
cy a
rea
Addi
ng It
Up
96
PEN
SIM
II t
o be
bui
lt in
resp
onse
to
criti
cism
.
Goo
d ba
sic
mar
shal
ling
of t
he k
now
n fa
cts.
Lack
of
dist
ribut
iona
l
anal
ysis
.
Crit
icis
m o
f cl
ient
gro
up
num
bers
.
Con
sulta
tion
still
ong
oing
thro
ugh
Gre
en P
aper
.
Ana
lysi
s bo
ught
in f
rom
DEM
OS.
Pens
ions
Pro
visi
on G
roup
brou
ght
toge
ther
larg
e
amou
nts
of a
naly
sis.
Sour
ces
as f
or P
ENSI
M
Als
o FR
S an
d G
HS
for
anal
ysin
g p
oten
tial
clie
ntgr
oup
s.
Econ
omic
mod
ellin
g
(PEN
SIM
) w
ith
besp
oke
anal
ysis
of
vario
us o
ptio
ns.
Pens
ions
pol
icy.
Ther
e ar
e a
num
ber
of
area
s w
here
fur
ther
deve
lop
men
t co
uld
be
unde
rtak
en.
In p
artic
ular
,
cons
ider
atio
n m
ight
be
give
n as
to
how
bet
ter
use
of t
he r
esea
rch
on
pric
e el
astic
ities
mig
ht b
e
inco
rpor
ated
into
the
anal
ysis
. Si
mila
rly li
nkin
g
up P
SM a
nd t
he D
ETR
mod
el m
ight
allo
w a
firm
er a
sses
smen
t of
the
inte
ract
ion
betw
een
priv
ate
and
soci
al r
ents
tobe
mad
e.
This
mod
el h
as b
een
used
to
qua
ntify
the
firs
t
roun
d ef
fect
s of
pol
icy
refo
rm o
n th
e
dist
ribut
ion
of in
com
e.
This
wou
ld n
ot h
ave
been
pos
sibl
e in
the
abse
nce
of t
he m
odel
.
The
mod
el is
fai
rly e
asy
to u
se d
esp
ite b
eing
deve
lop
ed o
ver
a sh
ort
time
scal
e.
The
mod
el is
ham
per
ed
by la
ck o
f lo
cal a
rea
data
.
The
mod
el c
anno
t sh
ow
beha
viou
ral i
mp
acts
i.e.
chan
ges
in h
ousi
ng
dem
and
in r
esp
onse
to
rent
s. T
he m
odel
has
no
links
bet
wee
n th
e so
cial
and
priv
ate
rent
ed s
ecto
r.
The
mod
el is
exp
osed
to
no e
xter
nal s
crut
iny.
The
mod
el u
ses
data
from
the
Eng
lish
Hou
se
Con
ditio
n Su
rvey
.
Stat
ic m
icro
-
sim
ulat
ion
mod
el
used
to
asse
ss t
he
dist
ribut
iona
l and
final
imp
licat
ions
of
refo
rmin
g re
nts
in
the
soci
al h
ousi
ng
sect
or.
Soci
al h
ousi
ng r
ents
.
Sug
ges
tio
ns
for
imp
rove
men
tSt
ren
gth
sLi
mit
atio
ns
Exte
rnal
in
pu
tD
ata
sou
rces
and
qu
alit
yTy
pe
of
An
alys
isP
oli
cy a
rea
Main Findings
from the
Case Studies
97
DfE
E do
es n
ot b
elie
ve
that
is p
ossi
ble
to id
entif
y
at a
n ea
rly s
tage
tho
se
who
will
be
liabl
e to
a
long
sp
ell o
f
unem
plo
ymen
t on
the
basi
s of
a f
ew in
dica
tors
.
But
othe
r co
untr
ies
do.
It
wou
ld t
here
fore
be
usef
ul
for
the
dep
artm
ent
to
mon
itor
whe
ther
thi
s
app
roac
h w
orks
, an
d if
it
does
, to
ens
ure
that
the
rele
vant
dat
a is
ava
ilabl
e
for
this
ap
pro
ach
to
bep
ursu
ed.
Stro
ng in
tera
ctio
n
betw
een
econ
omic
theo
ry a
nd a
naly
sis.
Goo
d us
e of
adm
inis
trat
ive
data
and
pilo
ts.
Ana
lysi
s p
laye
d a
com
ple
men
tary
rol
e to
pol
itica
l jud
gem
ent
but
did
not
in it
self
lead
toa
chan
ge in
pol
icy
dire
ctio
n.
Adv
ice
from
Pro
fess
or
Rich
ard
Laya
rd,
spec
ial
advi
sor.
Adm
inis
trat
ive
data
. A
naly
sis
of fl
ows
on
and
off
the
unem
plo
ymen
t
regi
ster
.
Labo
ur m
arke
t p
olic
y
in r
elat
ion
to jo
b
sear
ch a
ctiv
ities
.
Sug
ges
tio
ns
for
imp
rove
men
tSt
ren
gth
sLi
mit
atio
ns
Exte
rnal
in
pu
tD
ata
sou
rces
and
qu
alit
yTy
pe
of
An
alys
isP
oli
cy a
rea
Addi
ng It
Up
98
Cou
ld b
uild
on
FRS
data
to c
aptu
re b
enefi
ts s
ide
mor
e ac
cura
tely
, p
erha
ps
mer
ged
with
DSS
mod
el?
Mod
ellin
g of
tax
cre
dits
will
nee
d to
be
inco
rpor
ated
into
mod
el.
In lo
ng-t
erm
, ad
ditio
nal
labo
ur s
upp
ly r
esp
onse
s
to b
e m
odel
led.
In s
hort
term
, a
two
stag
e
app
roac
h co
uld
be u
sed.
The
first
rou
nd e
ffect
s
coul
d be
mod
elle
d as
at
pre
sent
. A
sec
ond
stag
e
wou
ld b
e a
sens
itivi
ty
anal
ysis
cen
tred
aro
und
diffe
rent
beh
avio
ural
assu
mp
tions
.
Use
r –
frie
ndly
and
is
rela
tivel
y op
en t
o p
ublic
scru
tiny.
Not
all
of b
ehav
iour
al
resp
onse
s m
odel
led.
Can
not
be u
sed
to m
odel
all a
spec
ts o
f bu
dget
ary
chan
ges.
Mod
el u
sed
by o
ther
dep
ts –
IR,
ON
S, C
&E
and
SO (
but
not
DSS
);
exte
rnal
aud
it w
ith IF
S
mod
el.
FES
– su
ffers
fro
m u
sual
pro
blem
s of
und
er-
sam
plin
g at
bot
h en
ds o
f
inco
me
dist
ribut
ion;
sm
all
sam
ple
siz
e co
mp
ared
with
FRS
. Ve
ry r
ecen
tly
been
ada
pte
d to
run
off
the
FRS.
Stat
ic m
icro
-
sim
ulat
ion
mod
el.
HM
T ta
x an
d
bene
fits
(IG
OTM
).
Sug
ges
tio
ns
for
imp
rove
men
tSt
ren
gth
sLi
mit
atio
ns
Exte
rnal
in
pu
tD
ata
sou
rces
and
qu
alit
yTy
pe
of
An
alys
isP
oli
cy a
rea
(i) USAA4.1 The team visited Washington toexplore the lessons that can be learned fromthe use of modelling and analysis in the US.The trip included visits to:
• Congress – General Accounting Office,Congressional Budget Office;
• Administration – Office forManagement and Budget, Department ofHealth and Human Services, Social SecurityAdministration;
• think tanks – The Urban Institute,The Brookings Institution;
• IFIs – World Bank and IMF.
A4.2 This note outlines the key observationsfrom the visit.
FindingsA4.3 The team found that the use ofanalysis and modelling in the US is moreextensive than in the UK and of muchbetter overall quality. In particular, datais more complete and the use of datamore sophisticated. The modelling donein government seems much closer to theleading edge of technical and academicadvance, and the staff are better qualified,typically holding PhDs rather than Mastersdegrees. There are three general reasons forthese differences:
• the conflictual nature of the USconstitution which places a premiumon the legislature providing well-foundedexamination of the executive’s proposals.The legislative requirement that everyproposal which is estimated to cost morethan $100m annually undergoes a formalcost benefit analysis is a powerful tool forensuring that the underlying analysis isaccurate. Each side consequently has anincentive to seek the very best possibleadvice;
• the level of resource devoted to thefull range of analysis from data collectionand publication to modelling withingovernment;
• the rich environment in which theanalysis and modelling takes place.External scrutiny of results is extensive,with a wealth of well-endowed think tanksas well as a vibrant consultancy sectorproviding challenge and support to allanalysis including that within government.
Direct lessonsA4.4 The US experience does not offer aprecise analogue to the UK but there area number of important points of directapplicability:
• use of existing data sources. TheUS relies on a similar mix of survey andadministrative data. The difference lies inaccess and the linking together of existingsources of information. Electronic accessto survey information is straightforwardencouraging use by researchers. And the
ANNEX A4. LESSONS FROM OVERSEAS
Lessons from
Overseas
99
links to administrative records are verygood, even outside government. This ismost clearly seen in the development ofMINT (the US equivalent of PENSIM)developed by the Urban Institute outsidegovernment uses survey data linked toincome data for participants from 1951to 1996. This gives a particularly strongsource for modelling retirement incomes.This process is receiving a new legislativepush from the Office for Management andBudget which is seeking to open updepartmental data bases to allow greatercross linkages and trying to increaseharmonisation of definitions andstandards;
• interaction with the academic sector.Government and academics have a gooddialogue. This is partly an unintendedby-product of the number of politicalappointees at the higher levels of the UScivil service. As a result there is a groupof people in the academic world with anongoing interest in keeping in touch withapplied analysis to inform their politicalambitions and constant interchangebetween the academic and the politicalworld. Beyond this, government is activein maintaining channels to the academicworld in a number of ways:
i. more use of contracting out both indata manipulation and modelling. Thisis partly a result of the greater numberof bodies working in these fields;
ii. inward academic secondments throughschemes under the government wideInter-Government Personnel Act. Therealso seems to be a greater willingness tosecond in academics in specific areas,for example to the Council of EconomicAdvisers where the current secondeesare doing modelling themselves;
iii. Departmental sponsorship of academicactivity, for example, the Departmentof Health and Human Services (DHHS)sponsorship of two ‘consortia’ toimprove research on old age issues.This is in its early stages but offers asource of academic support thedepartment can call upon;
iv. some of the departmental analystsare part of the academic world in amore formal sense, for example theCongressional Budget Office modellersare encouraged to publish in outsidejournals.
• anticipation of the forthcomingpolicy agenda. The system seemsparticularly responsive to gathering thedata needed for future evaluation andmodelling work. This is seen both inpreparing the ground for future welfarereform (at the DHHS) and in developingsurveys to understand behaviouralresponses (Social Security Administration).On the social policy side this may reflecta longer experience of policy whichemphasises the assessment of lifetransitions;
• the benefits of debate. The culturesurrounding the publication andavailability of data is a symbol of theopenness of the policy process. US policymakers and analysts see exposure offindings to debate as a positive benefitto the development of the policy process.However, although willing to consultwidely inside and outside government inthe development of models and settingtheir assumptions, departments seemsurprisingly unwilling to expose theirmodels to formal external scrutiny.
Addi
ng It
Up
100
Common problemsA4.5 Although there are lessons to belearned from differences of experience, thereare also some common problems on boththe technical and the institutional side:
• behavioural modelling is notsignificantly more advanced in the US(yet). However, some tentative steps arebeing made to deal with the data gaps,which underlie the modelling difficulties.In a number of areas there is more use ofdynamic micro-simulation modelling ingovernment, but results are currentlytentative;
• there are small area data problemsnotably in modelling the impact of manynational policy changes below the statelevel. This is because both state data arenot always adequate and because surveyscontain too few observations for anindividual state;
• the nature of models’ output is notalways well understood. The demandis for point estimates of policy changes notranges. This can be a problem since thenumbers attached to proposed policychanges by the Congressional BudgetOffice often carry a disproportionateweight in the policy process;
• time lags. Perhaps not unsurprisingly theUS is afflicted by the same problems as theUK with respect to measuring the impactsof welfare reforms. The problem arisesbecause of the time lags betweenimplementing a reform and having suitabledata to analyse the impacts of the policy.
A4.6 Specifically in the modelling area:
• there is little ex post evaluation ofmodels once a policy has beenimplemented. As in the UK, the impacts ofan individual policy change could notgenerally be separated from other effects;
• models go in and out of fashion.The tax benefit model run at the UrbanInstitute for a number of departmentshas had some periods when its fundingwas under close scrutiny. Researchand modelling budgets were also underconstant pressure at times of budgetarystringency. This is particularly importantsince there is little understanding of thelead times involved in constructing amodel;
• there is little external scrutiny of themodelling itself. There is little externalscrutiny of the modelling techniques; mostdebate, for example, in the CBO contextfocusses on the assumptions and theoutputs, but not on the modelling itself.
ConclusionA4.7 The overwhelming impression isof an analytical community that is valuedand necessary to the policy process.Although there are similar issues in detail onthe technical side, the context in which thequantitative analysis takes place is completelydifferent.
(ii) OECDA4.8 Team members also visited the OECDto develop an overview of the ways in whichother governments organise and accessinformation and analysis in a number ofpolicy areas.
A4.9 The main lessons to be learnt fromthe visit were:
• the issue of information availability andanalytical capacity in government iscommon across OECD countries;
• there is no one way of delivering adviceto the policy process. The preciseorganisational structure depends on thepolitical traditions of any particular country;
Lessons from
Overseas
101
• contestability in advice is a vital part of thepolicy process. Only by combining andconsidering several different points of viewcan the advice to underpin Governmentpolicy be fully tested;
• the Canadian approach to improving socialstatistics provides a practical example ofhow the centre of government canenhance or inhibit the improvement of theevidence base for policy (see box A4.1);
• joint planning of data collection,modelling and analytic needs is essential.
• in the employment sphere, the UK’s datasources are very much in line with normalpractice, although some other countriesare better;
• the US experience of profiling theunemployed with a view to targettinglabour market assistance has both practicaland organisational interest.
A4.10 The US showed well in terms of thestandard of data and analysis and Canada’scommitment to improving the informationavailable to policy makers at all parts of theprocess was particularly worthy of note.
Addi
ng It
Up
102
Box A4.1. Planning for Modelling in Canada
Canadian modelling is very highly regarded. The key to recent success in Canada has beento develop a joint strategy for data and modelling. This was in the first instance drawn upby a small group of high ranking civil servants – and the vision was initially unconstrainedby technology or practicality. Steps towards the vision were taken as opportunities arose,however, and in October 1996, a concrete and coherent forward programme of work wasdescribed in a paper by the policy group of Statistics Canada. This will include analysis ofareas as diverse as business performance and employee outcomes as well as labourincomes and family trends.
INTERVIEWSAdrian Gault – DTIAmanda Rowlatt – ONSAndrew Burchell – DETRAndrew Dilnot – IFSAndrew Oswald – Warwick UniversityAndrew Young – GADBruce Calderwood – DSSCaroline Rookes – IRCharles Tallack – PIU (DH secondee)Chris Shaw – GADCreon Butler – FCODan Murphy – DSS Daniel Instone – PIU (DETR secondee)Daniel Storey – HMTDave Barnbrook – DSS Denise Bagge – RIU, CODennis Roberts – ONSEdgar Jardine – Northern Ireland Statisticsand Research AgencyGeorge Kidd – Women’s Unit, COGraham Parker – Inland RevenueHolly Sutherland – Department of AppliedEconomics, Cambridge Hilary Jackson – HOJeff Jones – NAOJeremy Heywood – No 10Juanita Roche – CITU John Ashcroft – NAOJohn Bynner – Centre for LongitudinalStudies, Insititute for Education John Hills – LSE John Gieve – HMTJohn Thorpe – NAOJohn Vickers – Bank of England Jonathan Tross – CMPSJudith Littlewood – DETRJulian Jessop – LCDKaty Peters – HMT
Louise Dominian – SEUMalcolm Bradbury – Economist GroupManagement UnitMartin Weale – National Institute of Socialand Economic ResearchMichael Whitehouse – NAONick Buck – UK Longitudinal Studies CentreNick Lacey – NAONorman Glass – HMTPatricia Leahy – NAOPaul Gregg – HMTPaul Johnson – FSAPaul Randall – HMTPenelope Rowlatt – European EconomicResearchPeter Short – Inland RevenuePeter Waller – DTIPeter Grant – DFIDPhilippa Morgan – HMTRichard Bartholemew – DfEERichard d’Souza – DSSRichard Layard – LSE/DfEERoger Forder – MoDRon Amman – CMPSSharon White – No 10 Policy UnitStephen Jenkins – UK Longitudinal StudiesCentreTom Bentley – DEMOSUrsula Brennan – DSS William Price – HMTWilliam Kingsmill – DFID
OverseasAndrew Dawson – Washington EmbassyChristian Vergez – OECDDaniel Blume – OECDDon Oellerich – DHHSEdward Whitehouse – World Bank/AxiaEconomics
ANNEX A5.
Interviews
103
Josh Weiner – Urban InstituteGraham Hatch – IMFHenry Aaron – Brookings InstituteJoseph Minarik – OMBJustine Rodriguez – OMBLinda Bilheimer – CBOLoren Yager – GAOLynn Squire – World BankMark Pearson – OECDMyles Wickstead – DFID (World Bank) Norman Bowers – OECDPaul Van de Water – SSAPeter Hicks – OECD
Case StudiesC&E Smuggling – Derek Hodgson/Sue RoperCO Electronic delivery of Governmentservices -Jeremy CrumpDETR Air Quality – Martin HurstDETR Household projections – BruceOelman/Dorothy AndersonDETR Rent restructuring model – PhilipCox/Ray Kershaw/Luke BranderDETR National Road Traffic Forecasting – TomWorselyDfEE Labour markets – Bill WellsDH Long term care – Raphael WittenburgDH Waiting lists and times model – StuartKnight/John HalsallDSS PENSIM – John Ball/Chris CurryDSS pensions Green paper – MargaretPeirson/Guy Fiegehen/John Hughes/PhilipMorganDSS PSM- Jo Cockerham/Trevor Huddlestone FCO/DCMS PSA – Martin Williamson/AsifAhmed/Fiona Moore/Michael SeeneyGAD population projections- ChrisShaw/George RussellHMT IGOTM – Mark WardellHO crime model – Richard Price/Paul WilesLCD Legal aid model – Simon Hayllar/AndyMaultbyOFWAT efficiency models – FionaPethick/John Simpson
GLOSSARYBHPS British Household Panel SurveyCO Cabinet OfficeCITU Central IT UnitCORE Central Operational Research and
EconomicsCMPS Centre for Management and Policy
StudiesCSSB Civil Service Selection BoardCSR Comprehensive Spending ReviewCBO Congressional Budget OfficeC & E Customs and ExciseDfEE Department for Education and
EmploymentDFID Department for International
DevelopmentDAE Department of Applied EconomicsDCMS Department of Culture, Media and
SportDETR Department of Environment,
Transport and the RegionsDH Department of HealthDHHS Department of Health and Human
ServicesDSS Department of Social SecurityDoE Department of the EnvironmentDTI Department of Trade and IndustryESRC Economic and Social Research
CouncilEGMU Economist Group Management UnitEER European Economic ResearchEU European UnionECGD Export Credit Guarantee
DepartmentFES Family Expenditure SurveyFRS Family Resources SurveyFSA Financial Services AuthorityFCO Foreign and Commonwealth OfficeGEP General Expenditure PolicyGHS General Household SurveyGAD Government Actuary’s DepartmentGES Government Economic ServiceGORS Government Operational Research
ServiceGSR Government Social ResearchGSS Government Statistical Service
Addi
ng It
Up
104
HoP Head of ProfessionHSE Health and Safety ExecutiveHMT Her Majesty’s TreasuryHO Home OfficeIR Inland RevenueIFS Institute for Fiscal StudiesIGOTM Inter Governmental Tax and Benefit
ModelIGCB Inter-departmental Group on Costs
and BenefitsIFI International Financial InistitutionsIMF International Monetary FundJUVOS Joint Unemployment and Vacancies
Operating System LFS Labour Force SurveyLSE London School of EconomicsLCD Lord Chancellor’s DepartmentMAFF Ministry of Agriculture Fisheries and
FoodMoD Ministry of DefenceMINT Model of Income in the Near TermNAO National Audit OfficeNCDS National Child Development StudyNHS National Health ServiceNIESR National Institute for Social and
Economic ResearchNINO National Insurance NumberNRTF National Road Traffic Forecasting
ModelNTS National Travel SurveyONS Office for National StatisticsOFT Office of Fair Trading
OFGEM Office of Gas and Electricity MarketsOMB Office of Management and BudgetOFTEL Office of TelecommunicationsORR Office of the Rail RegulatorOFWAT Office of Water ServicesOR Operational ResearchOECD Organisation for Economic
Cooperation and DevelopmentOPA Output and Performance AnalysisPQ Parliamentary QuestionPIU Performance and Innovation UnitPSSRU Personal Social Services Research
UnitPAT Policy Action TeamPSM Policy Simulation ModelPSA Public Service AgreementRIA Regulatory Impact AssessmentRIU Regulatory Impact UnitSPRU Science and Technology Policy
ResearchSO Scottish OfficeSCS Senior Civil ServiceSSRA Shadow Strategic Rail AuthoritySEU Social Exclusion UnitSR Spending ReviewSERPS State Earnings Related Pension
SchemeSSP State Second Pensionvfm value for moneyWU Women’s UnitWTO World Trade Organisation
Interviews
105
1. Modernising Government. White Paper, Cm 4310 (HMSO 1999)
2. Professional Policy Making for the 21st Century. (Cabinet Office, September 1999)
3. Wiring It Up: Whitehall’s Management of Cross Cutting Policies and Services. PIU report(Cabinet Office, January 1999)
4. Building Trust in Statistics. White Paper, Cm 4112 (HMSO 1999)
5. Economic Appraisal in Central Government: a Technical Guide for Departments (HMSO 1997)
6. The Role of Economists in the Public Sector. The Business Economist: Vol 30 No 3 (1999)
7. Signs of Disintegration: A Report on UK Economics PhDs and ESRC Studentship Demand.S Machin and A Oswald (ESRC September 1998)
8. We all need pensions – the prospects for pension provision. (HMSO 1998)
9. Partnership in Pensions. (DSS 1998)
10. GSS Methodology Series. No 3 (HMSO 1996)
11. Partnership in Pensions: An Assessment by Richard Disney, Carl Emmersonand Sarah Tanner (IFS 1999)
LIST OF SOURCES
Addi
ng It
Up
106
9 780114 301613
ISBN 0-11-430161-1
Published by The Stationery Office and available from:
The Publications Centre (mail, telephone and fax orders only)PO Box 276, London SW8 5DTTelephone orders/General enquiries 0870 600 5522Fax orders 0870 600 5533
www.tso-online.co.uk
The Stationery Office Bookshops123 Kingsway, London WC2B 6PQ020 7242 6393 Fax 020 7242 641268–69 Bull Street, Birmingham B4 6AD0121 236 9696 Fax 0121 236 969933 Wine Street, Bristol BS1 2BQ0117 926 4306 Fax 0117 929 45159–21 Princess Street, Manchester M60 8AS0161 834 7201 Fax 0161 833 063416 Arthur Street, Belfast BT1 4GD028 9023 8451 Fax 028 9023 5401The Stationery Office Oriel Bookshop18–19 High Street, Cardiff CF1 2BZ029 2039 5548 Fax 029 2038 434771 Lothian Road, Edinburgh EH3 9AZ0870 606 5566 Fax 0870 606 5588
The Stationery Office’s Accredited Agents(see Yellow Pages)
and through good booksellers
£18ISBN 0-11-430161-1
© Crown copyright January 2000
The text in this document may be reproduced free of charge in any format or media without requiring specific permission. This is subject to the material not being used in a derogatory manner or in a misleading context. The source of the material must be acknowledged as Crown copyright and the title of the document must be included when being reproduced as part of another publication or service.
CABI 99-5263/0001/D16