research evidence and policy making

8/11/2019 Research Evidence and Policy Making

1/20

Administration, vol. 60, no. 2 (2012), pp. 119138

Research evidence and policymaking

in Ireland

Frances Ruane

Director, Economic and Social Research Institute1

Introduction

The growing international literature on policymaking processes, whichdraws on a range of different disciplines and perspectives, emphasisesthe importance of governance and the use of evidence forpolicymaking.2 The literature on evidence for policy has been welldeveloped in the UK, particularly in the health and education areas.3

There has been a growing emphasis on this topic in Ireland, with itgetting increased attention at conferences and in lectures over the pastdecade.4 However, many in the relevant research community5 did notengage actively with policy-related research or with policymakers until

very recently.6

119

1 This paper draws on the content of a lecture delivered at the Royal Irish Academy

(RIA) to mark the fiftieth anniversary of the Economic and Social Research Institute

(ESRI). It has benefited from comments from attendees, from colleagues at the ESRIand from a number of policymakers who read earlier drafts. The views expressed are

personal to the author and should not be interpreted as reflecting the views of the ESRI.

The usual disclaimer applies.2 Some of this work involves the creation of data sets that can support further research.

See, for example, Hardiman et al. (2012).3 The ESRC Centre for Evidence Based Policy and Practice has provided a focus for

these discussions in the UK, where much of the emphasis has been on health and

education. See Young et al. (2002).4 See, for example, National Economic and Social Forum (2007) and the papers by

Ruane & Whelan (2011) and Hearne & Watt (2011).5 These are social science researchers primarily based in higher education and research

institutions.6 There have been several policy conferences since 2008 and these have been well

attended by both researchers and policymakers. As time has progressed, the range of

topics has widened from the macroeconomics and finance area to a range of other policy

areas.


2/20

The need to ensure social cohesion in the face of difficult decisionsdemands a robust policymaking process. Using research evidence to

help improve policymaking has much to commend it, especially astolerance for ineffective policies and wasteful use of resources is nowvery low.7 This paper looks at the research/policymaking interface.

The paper draws on personal experience in the research andpolicymaking space.8 For much of the past three decades, Irish policydesign appears to have been dominated by administrative concerns,

with less attention given to economic and social factors. The one majorexception to this was the set of programmes supported by EUStructural Funds, where strong planning and evaluation processes

were put in place. The move away from robust ex ante and ex postevaluation coincided with the ending of these projects. This, combined

with Irelands exceptionally poor record in physical planning(especially during the years of very fast growth, referred to below asboom years),9 has left us with major policy liabilities that needattention.10 Additionally, with some notable exceptions, there waslittle attempt in the early 2000s to measure robustly the impact ofpolicy and little obvious commitment to the use of data/analysis toinform policy within line departments.11 Furthermore, until recenttimes, many stated policy objectives were poorly specified, and policyinterventions often conflicted with these objectives.12

This paper is relevant to research in areas most directly linked topolicy economics, sociology, social psychology, political science,

120 FRANCES RUANE

7 This issue has been receiving increased media coverage since 2008/9.8 This reflects over thirty years of engagement in different capacities. It also takes into

account my present perspective as Director of the ESRI, whose mission is to deliverresearch that is relevant to policy.9 The National spatial strategy (NSS) was launched in 2002, after a decade of rapid

growth, which had been supported by the EU Structural Fund Programmes. The NSS

appears to have had little positive impact on national physical planning in the mid 2000s.

Specifically, there seems to have been little coherence in the relationship between

housing, transport and land use. Furthermore, the gateway towns identified in the NSS

were ignored in the governments decentralisation plans, which were announced after it

had been published.10 The built environment and the sustainability of some activities that were funded when

money was more plentiful (e.g. higher education) are particularly important areasrequiring attention.11 The transfer of the Department of Finances evaluation unit to Tullamore as part of

the decentralisation plan was seen by many as providing a signal that such activities were

not being prioritised.12 Specific examples can be found in the original RIA lecture in ESRI Working Paper

No. 354 (Ruane, 2010).


3/20

geography, demography, management, etc.13 However, and whilerecognising that multidisciplinary approaches are often required, the

papers main focus is on economics, as concepts and frameworks fromthis discipline are of particular relevance in the present climate.On the policymaking side, the focus is on those civil and public

servants who analyse policy possibilities and present options(sometimes accompanied by recommendations) to politicians. Theterm policymaker is used to refer to them, following common usage ofthat term in Ireland.14 The paper does not discuss the policy interfacebetween senior civil/public servants and politicians. Clearly, the

attitude of politicians towards the use of evidence to inform theirdecisions impacts on the emphasis that is placed on such evidencewithin the civil/public service. Finally, the paper does not discusseither the engagement between researchers and politicians or the rolesplayed by researchers in policymaking bodies.

This discussion is set against a background where major policychallenges must be addressed if Irelands economy is to get back ontrack and social harmony is to be maintained. Even the most cursoryreview of the rapid increases in current spending over the boom yearssuggests that value for money played an inadequate role in decision-making.15 This was reflected in the growing gap between the rhetoricand reality in the late 1990s/early 2000s in relation to evidence-basedpolicy and evidence-based approaches to evaluation.16 Over thisperiod, money was spent on many nice to have rather than need tohave projects and programmes, with little apparent regard to

Research evidence and policymaking in Ireland 121

13

In practice, many policies cover areas where the content of other disciplines is highlyrelevant, such as medical science, environmental science and engineering.14 To people from outside Ireland, this term may look like a misnomer. However, it has

become common parlance in Ireland over the decades to refer to policymakers and

politicians. This term is not seen as denying in any way the constitutional role of

ministers (and the Dil) in the policy process, but it recognises that ministers bring

forward policies that derive strongly from options put forward by the policymakers.15As revenues were pouring into the exchequer, research or analyses that were critical

of policy were not welcome, and criticisms of some individual policies on the basis of

poor value for money (e.g. the Western Rail Corridor) were seen as petty. The

availability of revenues worked against cost being taken systematically into the decision-making, as it would at a time of scarce resources.16 Various presentations (including those from the Department of Finance) identified

the need for a more evidence-based evaluative approach but they were ignored.

See http://www.dcu.ie/education_studies/ien/iendata/David%20Doyle%20Keynote%

20Speaker.ppt; http://www.dcu.ie/education_studies/ien/iendata/FRuane%20 Evaluati

on%20Conference%20Presentation.ppt.


4/20

evidenced-based prioritisation in the decision-making process. Forexample, there is no evidence of research or analysis to inform the very

significant decision made in December 2003 to decentralise large partsof the civil/public service.17 Indeed, it appears that this decision alsolacked any prior consultation with the senior public servants who

would have been able to provide advice if asked.Todays tough decisions need to be grounded in evidence if they are

to command respect, acceptance and support. The researchcommunity can contribute to this process so that evidence-informedpolicymaking can become a reality. Recent policy initiatives ingovernment departments, led by the Department of PublicExpenditure and Reform (PER), have signalled support for thischange.18 These are referenced further below.

The remainder of the paper addresses four questions:

i. What can research offer to the policymaking process?ii. Why is greater use not made of research in the policymaking

process?iii. Why is more policy-focused research not available?iv. What is needed for a more productive engagement between

researchers and policymakers?

What can research offer to the policymaking process?

Policymaking is complex, reflecting historic patterns, institutionalstructures, operational legacies, cultural influences and internationalconstraints. The policymakers role is to bring forward options for

political decision-making. The supporting analysis may be prepared byanalysts within the system or occasionally by consultants.

122 FRANCES RUANE

17 This contrasted with the UK decision in 2004 to decentralise the civil service further

out of London. This was based on a major government report, which focused on

a careful cost analysis and on plans to build regional labour markets that could

support skilled labour markets for public sector jobs. See http://www.hm-

treasury.gov.uk/consult_lyons_index.htm.

18 For example, in the Comprehensive Expenditure Report of December 2011, PER setout a range of reforms, including a unified and updated Value-for-Money (VFM) Code

(to provide for appraisal (ex ante) and evaluation (ex post) of current as well as capital

expenditure), the creation of a Public Service Evaluation Network to provide a forum

for advancing the evidence-based policy agenda and a reformed medium-term budget-

formation process. Some of the thinking that informed this process is set out in Hearne

& Watt (2011).


5/20

Researchers can contribute most to the policy-formation processwhere complex issues require comprehensive frameworks/models to

be developed or new empirical evidence to be analysed. Bothresearchers and analysts need to understand from the outset that anygiven piece of research is unlikely to provide a silver bullet solution,and that the key role of the researcher is to draw out the implicationsof the theoretical or empirical analysis for policy design or evaluation.

How can research help with policy design?

Economic theory can play an important role in clarifying objectives,assumptions, rationale and mechanisms. For example, it can helpidentify how policy can generate perverse incentives, resulting inbehaviour that is opposite to stated policy objectives. There have been

various cases where this has been seen to arise: university lecturers donot focus on teaching if their promotion prospects depend solely onresearch publications; patients do not use GP services if outpatientcosts are free and GP visits cost 50; government agencies do not usefunds efficiently if they know that their budgets will be cut if they haveunspent money at year end.19

For new initiatives or major changes in the direction of policy,20

researchers can provide additional independent, critical reviews ofempirical evidence both national and international.21 This maysimplify the policymakers decision or may complicate it by drawingout elements that could otherwise have been ignored.22 To take aninternational example, research in macroeconomics and finance inrecent years shows that researcher expertise has made a significantcontribution but has not necessarily helped to make the policymakers,

or indeed the politicians, immediate job any easier.Researchers can generate evidence from existing or new data. For

example, in designing tax changes, existing data can be used tomeasure likely responses. In some cases, policy experiments can beundertaken to explore the impact of specific interventions, e.g.


19 This latter problem is now in the process of being dealt with as part of the recent

reforms, i.e. the introduction of multi-year expenditure ceilings, with allowance for

carry-over of unspent sums into the following year.20 Where the literature is well developed and there is considerable research evidence

available, the researcher may provide little additional value and, for the most part, it will

be appropriate to undertake much of this analysis within PER as it builds up its skill sets.21 See, for example, the two research reports prepared for the Expert Group on

Resource Allocation and Financing in the Health Sector (Brick et al., 2010).22 In so doing, it may reduce future costs and difficulties.


6/20

through randomised control trials. Analysis of longitudinal dataprovides a better basis for judging policy impact. The current

investment in longitudinal studies of children and the elderly areexamples of good national research planning that will provide muchbetter evidence in the future to inform policy. It will also provide datafor high-quality academic research.

The time horizon for good policy research can be very long. Todaywe are reaping the benefits of decisions made more than a decade agoto invest in our statistical infrastructure. These included new surveysby the Central Statistics Office (CSO), such as the Quarterly NationalHousehold Survey and the Survey of Irish Living Conditions,23 and theengagement of each government department with the CSO in relationto obtaining statistical data from administrative records. The recentgovernment decision to extend departmental strategies for data toinclude research is a very welcome development, and should help tomake sure that maximum return is generated by expenditure on policyresearch.24

How can research help to measure the effectiveness of policy?

Ensuring that decisions today are effective and efficient is crucial. Thismeans undertaking ex ante social cost-benefit analysis of capitalprojects and regularly evaluating current programmesex post to see ifthey are meeting their stated objectives. In both cases, theoreticalinsights help link policy objectives to policy actions and help definecoherent frameworks to measure their effectiveness. The use ofprogramme logic models is helpful in this regard and these are nowbecoming part of the Irish policy-evaluation system.25 However, some

areas require much more sophisticated analysis, with precisemeasurement of the counterfactual in order to explore causalrelationships properly and ensure that the use of evidence is

124 FRANCES RUANE

23 This important study had its origins in the Living in Ireland Survey, which was

undertaken by the ESRI and funded in large measure by the European Commission.24 The report by the Research Prioritisation Steering Group (2012) identified research

for policy as a specific area of research supported by government, different from basicand applied research. It noted that much commissioned research was ad hoc rather

than part of an overall programme, and that peer review could assist in improving its

quality (pp. 3940). An example of a departmental data and research strategy can be

found at http://www.dcya.gov.ie/viewdoc.asp?fn=/documents/Publications/ndrs.htm.25 The VFM Code (http://vfm.per.gov.ie) sets out this model, which connects inputs,

activities, outputs and results, and defines their linkage to strategic objectives.


7/20


8/20

of their own thought processes and experiences, a danger which thepast decade illustrates only too well in many OECD countries.

Why is greater use not made of research in the policymakingprocess?

To understand how greater use might be made of research in thefuture, it is helpful to ask why, given the potential contribution fromresearch, so little use has been made of it in the past, and specificallyin the decade 19982008. Policy in this period was dominated by theprogramme for government agreed by coalition partners and by thesocial partnership framework. Major policy issues were decided insettings where there was great pressure to obtain consensus and hugemedia attention. The focus was on the decision-making process (the

winners and losers) at the expense of the content of the agreement. Adecision from such a process could come to be implemented even ifthere had been no prior analysis or rigorous costing undertaken.32 Thismeant that some policy decisions did not involve the policymakers atall, and their role in the process was simply to implement and evaluate.What can be learnt from this? Unless analysis and research arepubliclyavailable to inform the programme for government process, theirability to inform policy is quite limited.33 This means that unlesspolitical parties indicate the policies they are considering well inadvance, the required analysis and research are unlikely to beavailable.34

In contrast, the social partners had an opportunity to draw onresearch evidence through the underpinning work of the National

Economic and Social Council (NESC). However, over the course ofthe 1990s, the role of the NESC increasingly became one of supportingthe partnership process by producing consensus documents, ratherthan one of publishing objective policy analyses (a role partiallyfulfilled by the National and Economic Social Forum). These

126 FRANCES RUANE

32 This issue was addressed in the Wright report of December 2010 (Independent

Review Panel, 2010). It described the budgetary process as being overwhelmed by the

combination of the programme for government and social partnerships systems.33 Even if available, the analysis/research may be legitimately ignored by a

democratically elected government.34 It has been suggested that ahead of New Labour taking power in the UK in 1997, the

Treasury had analysed and evaluated all of the policies it had been promoting in the

previous two years. This analysis provided evidence for Labour of what might or might

not work. Of course, there were no coalition complications in that setting.


9/20

consensus documents contributed significantly to industrial peace andpartnership harmony. However, the by-product of this was that the

policy arena had fewer analytical documents that drew on national andinternational research evidence to set out the hard policy challenges,choices and their consequences.

Another institutional factor impacting on the use of research wasthe balance in analytical expertise between the civil service and the

wider public service. Whereas in most countries policy details aredesigned by specialists within government departments, in Ireland thespecialist knowledge became fragmented across a growing number ofspecialist agencies, and in some cases within private sectorconsultancies. As a consequence, many government departments hadlittle specialist knowledge or resources to analyse and develop policy.This further reduced the likelihood of research being used, with theresearch agenda falling between agencies and departments. This oftenled to potential for duplications and omissions, a problem that will bereduced in the future if departments implement coherently their dataand research strategies.

The application of benchmarking is a positive feature of Irishpolicymaking, when done objectively and rigorously.35 It can be usedto challenge thinking and inspire action if the benchmark is the bestor the most relevant. However, even a good benchmarking exercisemust be viewed as the starting point rather than the end point forcomparative analysis, and policymakers must realise that comparativeresults can be misleading, unless they are set in a properly definedframework. With the exception of the original benchmarking report onpublic sector wages and salaries in the early 2000s,36 Ireland generally

has conducted this type of benchmarking very openly and trans-parently. In recent times, developments in the websites of somegovernment departments, agencies and public bodies in Ireland haveincreased transparency rapidly. These developments include reports,statistical data, organisation charts and, in the case of one departmentat least (PER), the minutes of its management meetings (with a two-month lag).

A less-positive feature is the extensive use of expert groups

operating over relatively short time periods, often required to developmajor strategy positions. While other countries at our stage of


35 The Competitiveness Council is an example of a body that has used international

benchmarking to challenge actions and policies.36 Benchmarking can be seen as the low waterline in Irish policymaking, with a

deliberate decision made not to place the evidence in the public domain.


10/20

economic development use such groups from time to time, Irelandseems to use them more frequently. What exactly is their role? Are

they intended to generate new ideas, to challenge conventionalthinking, to build more policy coherence or to develop a widerconsensus? How independent are the experts and how do they makeuse of existing research evidence? What is the quality standard forthese reports? Might it not be more useful and more transparent todevelop expertise within the relevant departments or agencies andthen issue published reports for wider consultation? What should theofficial response process be to such expert reports? The extent of theuse of expert groups suggests that a systematic review of their impactand effectiveness would be very informative.37

Ireland has often called on the OECD for assistance. When andwhy do we do this? Is it the lack of local expertise or desire to get theOECD to deliver tough messages? The status of an OECD reportcarries an exceptional amount of authority along with a presumptionthat its recommendations should be accepted without further debateor question.38 The quality of reports depends on the skills of the team,the validity of the international comparators, the terms of referenceset by the Irish policymakers and the quality of Irish research that canfeed into their work. Even if the report is excellent in its own terms,there is still a need for a real debate on its conclusions andrecommendations before they are accepted or ignored. It would behealthier if the role of the OECD were to become one of challengingus and helping us to identify possible policy solutions rather thanproviding us with the solutions.

Today, the troika is an additional source of external expertise. The

relationship between Irish policymakers and the troika is clearlydifferent to that between Ireland and bodies such as the OECDbecause of the troikas specific oversight role. Nevertheless, there is apossible gain from that engagement, notwithstanding the desire forthe policymakers to bring it to an end at the earliest possible date.

128 FRANCES RUANE

37 Their costs often greatly exceed what is measured by the departments that set them

up; in particular, no account is taken of the opportunity cost of the participants.

Furthermore, their time frame often means that only very narrow research questions canbe addressed and that commercial consultancies are the only bodies in a position to

undertake the work. This increases the costs of the expert group and reduces the

likelihood of knowledge transfers or of the analysis generated having future value.38 While the calibre of OECD reports may be much higher than other local consultancy

reports, the latter do not enjoy the same status. However, unlike most other consultancy

reports, those by the OECD are always published.


11/20

There is a striking similarity between the content of the reform agendaset out in the troika documents and what is contained in the National

recovery plan 20112014,which was published before the arrival of thetroika. Consequently, the troikas endorsement of that agenda, basedon analysis and international evidence, is effectively assisting thegovernment in tackling a range of powerful vested interests.

In considering the use of evidence, one should note the differencebetween a process that uses independent, comprehensive and rigorousanalysis of the evidence to inform policy and a process based on theselected use of external reference points (either research or policy)

without any rationale. Adopting a policy because it seems to work wellin country X is not prudent, especially when the policy setting isdifferent. The effectiveness of such a policy should have been fullyevaluated before being considered for adoption in Ireland.39

A final explanation of Irelands low use of research in policymakingmore generally may be the poor respect for, and knowledge of,statistics. This is, in part at least, due to how the education curriculumat second level is delivered, with few candidates covering the statisticalsections of their mathematics courses.40 This inhibits people engaging

with research results in the policy arena. Ireland is a nation ofstorytellers and we all like good anecdotes; statistics leave people cold.The research community could help overcome this by producingstatistically valid examples to illustrate findings. Otherwise, anunpopular finding coming from high-quality research can be immed-iately trumped by an unrepresentative anecdote, as most people arenot in a position to judge the difference between the representativeand unrepresentative anecdote.41 Research findings could be more

informative if researchers, having undertaken robust analysis, improvehow they present research to the wider public.42 While it is challengingto simplify without being misleading, it would also assist journalists inproviding good material for the general public.


39 For example, there was much discussion about introducing the Dutch health

insurance system in Ireland while evidence was emerging from the Netherlands that it

was not yielding the expected results.40 Recent changes in the mathematics curriculum under Project Maths should make a

contribution over coming decades but, on its own, the project will take a long time toimprove overall statistical literacy.41 The comment I dont deal with statistics, I deal with real people is often uttered with

pride in reality this means I dont deal with real people, I deal with a few people I

know.42 The mixed methods used in sociology play this role very effectively they combine

statistical analysis with qualitative analysis that provides case-study type examples.


12/20

Why is more policy-focused research not available?

In recent years academic institutions have increasingly emphasised the

importance of peer-reviewed, international journal articles inpromotion processes. While this emphasis has merit, it leaves littleincentive for academics to undertake policy research. This is not justan Irish problem. In the UK the immediate impact of the research-assessment exercise was a dramatic reduction in the volume of policy-relevant research in the 1980s.43 In response, the UK Governmentestablished a system of funding research programmes that are relevantto policy, mostly in cooperation with the Economic and Social

Research Council.44

While this has yielded a positive response, it isnoteworthy that much of the policy research is being undertaken bythe older generation of researchers, who are less concerned withpromotion. These older academics have also built up the institutionalknowledge necessary to undertake good policy research.45

Up to now, there has been no formal system for funding policyresearch across the full policy spectrum in Ireland. However, there hasbeen public funding for programme-based social science research and

for programmes in specific policy areas. Examples of the formerinclude the creation of institutes in the universities, e.g. the GearyInstitute, University College Dublin (UCD); the Institute forInternational Integration Studies, Trinity College Dublin (TCD); andthe National Institute for Regional and Spatial Analysis, NUIMaynooth. Examples of the latter include funding for policy researchin Teagasc, the ESRI and the IPA (from the Committee for PublicManagement Research).46 The approach varies widely acrossdepartments and agencies, as does the balance between project and

programme funding, and between research and consultancy.47Philanthropic funding for policy research, very common in the US, hasbeen modest in Ireland, with some striking exceptions, most notablyThe Atlantic Philanthropies. Its support for research on children and

130 FRANCES RUANE

43 This exercise sought to measure research outputs of universities, which then became

the basis for funding higher-education research.44 This involved setting up research centres and clusters with a specific policy focus.45 One countervailing factor is the growth in the number of new policy journals. This has

increased the willingness of university academics to undertake policy research. InIreland The Economic and Social Review, the main local academic journal for economics

and empirical sociology articles, introduced a policy section in 2009.46 See http://www.agresearch.teagasc.ie/rerc/research.asp, http://www.esri.ie/research/

research_areas/ and www.cpmr.gov.ie.47 The ESRI is among a number of bodies that directly receive funds to support research

for policy, and its research focus reflects where such funds are available.


13/20

on ageing has demonstrated the research communitys willingness andability to respond when funding is available. In certain policy areas,

most notably health and children, charities have also funded research.In contrast to some of the larger OECD countries, there is negligiblepublic good research funded by the private sector in Ireland.48 Interms of policy-relevant research, Ireland has benefited from EUfunding, which has mostly supported cross-country comparativeresearch.

Yet another reason for the low level of independent policy researchduring the boom years was the widespread use of consultancy reportsto support policymaking, often where terms of reference ensured that

the outcome was virtually predetermined. The absence of budgetconstraints and the increased emphasis on individual accountabilityreinforced that pattern, as the reports could provide apparentlyindependent support if required and otherwise were typically notpublished.

One very positive development in the past decade is the availabilityof high-quality Irish data, especially from the CSO. With(anonymised) micro data, research that was previously not possible

can now be conducted, and this research has international journalpotential. Given the states large investment in data collection, it isimportant that these data are being used in research.49 Their use hasbeen slower than might have been expected, given the growingnumbers of academics and PhD students over the past decade. Thisreflects the considerable investment of time required to use any givendata set and often the need for teams with mixed skill sets to realisethe potential benefits. The availability of these data enhances the

possible synergies between academic and policy research. Whileresearchers typically focus on the quality required for journal articles,policy research needs to be just as robust as academic research andrequires a much greater investment in institutional knowledge. Somepolicy research in Ireland is subject to peer review, with departmentsand agencies sending out draft reports to independent researchers forcomment.50


48 The ESRI has received some funding of this type for research but the general view of

the private sector is that it is the governments responsibility to fund policy research.49 Many of these anonymised data sets are available in the data archive at UCD.50 For example, the ESRIs review process involves seeking comments from internal and

external peers, as well as seeking comments from policy specialists in relevant agencies.

Frank Barry (TCD) recently suggested that one method of improving the quality of all

commissioned research by government (including that prepared by consultants) would

be to involve a seminar to peers as part of its process. In a similar vein, PER (contd.)


14/20

One of the possible silver linings in todays dark clouds is thegrowing numbers of young academic researchers prepared to engage

in policy debates. This followed on from a decade of almost noengagement with a younger generation of researchers. One can behopeful that this will translate over time into increased research acrossthe full range of policy issues.

What is needed for a more productive engagement betweenresearchers and policymakers?

The relationship between the research and policy communities

depends on the networks linking the individual researcher(s) andpolicymaker(s). When the two communities in Ireland were very muchsmaller, such networks emerged naturally; this does not happentoday.51 Until 2011, when the incoming government committed tobuilding up specialist skills in the civil service, the policy of creatingstrong generalists meant that there was little reason for engagement.Relationships that had existed were continuously undermined by themobility of policymakers across areas, usually being replaced by

people with no specialist knowledge of the area and with a different setof analytical skills. The increased specialisation in research in mostdisciplines over the past two decades has compounded the problem.

Some ways of increasing productive engagement betweenresearchers and policymakers are addressed below for each groupseparately and both together.

Researchers

Researchers can increase the accessibility of policy-relevant researchto policymakers by providing non-technical summaries/abstracts ofresearch findings, e.g. via research bulletins, policy briefs, web pagesand blog postings. This could cover research relevant to Ireland thathas been published internationally and is not readily accessible topolicymakers. Working papers could also have such non-technicalsummaries, as could PhD theses undertaken at Irish universities.52 It is

132 FRANCES RUANE

50 (contd.) has indicated its intention to publish economic evaluations in the future; this

is a natural extension of recent publications of reviews on its website.51Academic programmes, such as the MSc in Economic Policy Studies at TCD and the

various programmes offered by the IPA (more recently in conjunction with UCD), have

provided the main links into the policy community over the past decade.52 Such summaries could be encouraged by research institutions and made mandatory

for research that is funded from the public purse (e.g. Irish Research Council for the

Humanities and Social Sciences).


15/20

arguably a good discipline for researchers to summarise findings in anaccessible form and, where appropriate, draw out their policy

relevance. The benefits to the policy community and to students couldbe considerable. As noted above, economists engaged directly inpolicy research could identify examples to illustrate the generality oftheir findings.

Possibly the greatest challenge for researchers wishing to engage inpolicy research is the scale and nature of the issues, which requiremulti-person and increasingly multidisciplinary teams. While thesocial sciences are moving towards more team-based research, thetradition of the single scholar is still strong, as is the tradition of thesingle discipline. Moving to multidisciplinary teams takes time andeffort, especially if programmes in the higher education system arehighly specialised.53

What can research institutions do to support better engagement?They can demonstrate that they value policy research by includingpolicy engagement in the criteria for promotion (under pro bonoheadings). They can signal support through a positive attitude towardspostgraduate research and teaching programmes in policy areas.

Finally, they can ensure that their accounting systems and intellectualproperty contracts do not hinder the development of policy research.

PolicymakersWhile the individual, rather than the institution, is the driver on theresearch side, the opposite is the case on the policymaking side. Theindividual policymaker can do very little to promote greaterengagement with the research community without institutional

endorsement. Consequently, the culture towards research within adepartment or agency has a major influence on the behaviour of itspolicymakers. So, what might help to stimulate greater engagement

with the research community?It is something of a paradox that, in a country where public

discourse for two decades has emphasised theknowledge economy andknowledge society, the civil service has, until very recently, pursued ahuman resource strategy that effectively penalises people withspecialist knowledge by either narrowing their career prospects orforcing them to deskill. What is needed is a combination of generalistsand specialists. This means developing and retaining the specialistskills of people entering the civil service.


53 High levels of specialisation at undergraduate level result in little understanding of

the approaches and contributions of different disciplines.


16/20


17/20

planned data and research strategies, these should increase thebenefits of research, especially on cross-cutting issues, and should give

greater assurance that funds for policy research are productively andcoherently spent.

Policymakers and researchers

What extra can be achieved by both groups fostering the engagement?Exchanges of personnel between the two sectors should be promotedand welcomed, but these will take some time to develop. Meanwhile,

working closer together would help establish a clearer understandingof roles, interests and the requirements of researchers andpolicymakers.59 This would allow more productive interactions,thereby enhancing benefits and reducing potential tensions. It wouldhelp in turn to foster commitment on both sides to building ameaningful research agenda, which would see researchers producingpolicy outputs (published reports) and academic outputs (peer-reviewed papers).60 Interactions with policymakers could stimulateresearch into new issues and, when linked to pilot studies involvingmultidisciplinary teams, could generate significant new research

possibilities.By combining efforts, researchers and decision-makers would

inform, support and promote societal well-being in the face ofpowerful, vested interest groups. The potential beneficiaries of manypolicy improvements may be widely dispersed and without theresources to promote such policies, e.g. a patient-centred healthsystem. In contrast, some policy changes, such as the abolition of thebank levy in Budget 2007, benefit a small number of institutions at the

expense of the general taxpayer.61 Ireland is good at bringingstakeholders to the table. But this process needs to involve challengingthem with robust evidence and analysis so that they engage with thetotality of issues rather than simply their own agendas. If thestakeholder process does not involve challenge, it may simply lead to


59 This involves understanding incentives and gaining clarity of and respect for roles and

governance by both sides.60 If the researchers have no real commitment to policy research, the benefits will be

small in policy terms and the major benefit may be in terms of contributing to

institutional (university) overheads. Similarly, there is no point in policymakers

appearing to embrace research while believing that they know the answer already and

really have nothing to learn.61 The banks were in a position to lobby the taxpayers were not.


18/20

no real engagement and possibly to solutions that are beneficial tothose at the table at a cost to society at large.62

The engagement is not without possible risks and the mostobvious is that of promoting group think, especially if people wereonly comfortable with consensus. One way of helping to avoid this isby having open dialogue events, under the Chatham House Rule, andgreater acceptance of the value of critical thinking on both sides.63 Ifthe debate is one-sided either with the researcher pronouncing andthe policymaker silent, or with the policymaker picking holes and theresearcher disengaging there will be no benefits from theengagement. These Chatham House events will not happen unlessthey are organised by some body or group and supported byresearchers and the leadership in the policy community.

The other large risk is that the researchers lose, or are perceived tolose, their independence, by engaging with the policymakers. To avoidthis, an open and transparent process is needed. As part of thisprocess, policymakers would provide inputs to the research that seekto inform and improve the analysis rather than influence theconclusions. Similarly, researchers must engage with thesepolicymaker inputs and with standard peer-review processes. Finally,the researchers must retain their right to publish the policy researchand commit to it being placed in the public domain so society at largecan examine the findings.64

Concluding comments

This paper has explored some ways in which we might foster greater

engagement between researchers and policymakers in Ireland today.This engagement would increase the return on the national investmentin research that is relevant to policy.65 For the engagement to

136 FRANCES RUANE

62 The discussions around the renegotiation of the medical consultants contract in

200608 would appear to be one such example.63 Such events were relatively limited in the past decade, and meetings of the Statistical

and Social Inquiry Society, which have provided a forum for policy debates for over 150

years, have been poorly attended by policymakers in recent years.

64 In very exceptional cases, this may not be practicable for confidentiality reasons, butit should be the default, with the arguments for the exception clearly established. The

requirement to publish is a standard condition in ESRI contracts for all its

commissioned research.65As well as engaging with current policies, independent researchers are well placed to

contribute to identifying future problems and complexities, because of their longer time

horizons.


19/20

contribute to making the policy process more research- and evidence-based, strong support is needed from across the full political system.

In a context where persistent inefficiencies in policy design anddelivery are now less acceptable, and definitely unaffordable, evidencefrom research can help to

ensure that people understand why tough decisions arenecessary;

demonstrate that tough choices can be fair; and show that lessons have been learnt from the past.

Recent developments point to possibilities of better engagementbetween researchers and policymakers in the future. However, this willnot develop automatically unless it is welcomed and endorsed by thepolitical system and by public-sector leadership.66 It also requires thatresearchers who are interested in policy invest in understanding themany recent changes in institutional systems and structures. TheIGEES will take time to build up its capacity and capability, andinterim measures will be required to speed up the engagementprocess.

Despite the many weaknesses of policymaking over the boomyears, there were some good investments made that are relevant topolicymaking.67 These included investments in the data infrastructure(a sine qua non for empirical analysis) and investments in humancapital (increase in the number of MSc and PhD students graduatingfrom universities and in the numbers of civil/public servants pursuinghigher degrees). Furthermore, most departments and agencies have

had experience of investing in research to inform policymaking, albeiton a somewhat ad hoc basis. What is required now is that the approachto research becomes more systematic, more questioning, and with agreater focus on the important policy questions facing Ireland today.

References

Brick, A., Nolan, A., OReilly, J., & Smith, S. (2010). Resource allocation,

financing and sustainability in health care: Evidence for the Expert Group onResource Allocation and Financing in the Health Sector (Vols III). Dublin:Department of Health and Children & Economic and Social ResearchInstitute.


66 This means support by public representatives generally as well as by ministers.67 There were good investments in many other areas also, e.g. parts of the road network.


20/20

Hardiman, N., MacCarthaigh, M., & Scott, C. (2012). The Irish stateadministration database. Retrieved from http://www.isad.ie [5 July 2012].

Hearne, E., & Watt, R. (2011). Research and policy-making Strengtheningthe link.Journal of the Statistical and Social Inquiry Society of Ireland, XL,15266.

Independent Review Panel. (2010). Strengthening the capacity of theDepartment of Finance: Report of the Independent Review Panel. Dublin:Department of Finance.

National Economic and Social Forum. (2007).Evidence-based policy making:Getting the evidence, using the evidence and evaluating the outcomes. Dublin:National Economic and Social Forum.

Prendergast, C. (2011). What have we learnt about pay for performance?The Economic and Social Review, 42 (2), 11334.

Research Prioritisation Steering Group. (2012). Report of the ResearchPrioritisation Steering Group. Retrieved from http://www.forfas.ie/media/ffs20120301-Research_Prioritisation _Exercise_Report.pdf [5 July 2012].

Ruane, F. (2010). Research and policy making (Working Paper No. 354).Retrieved from http://www.esri.ie/UserFiles/publications/WP354/WP354.pdf [5 July 2012].

Ruane, F., & Whelan, B. (2011). Building research capacity in the social

sciences Alternative approaches. Journal of the Statistical and SocialInquiry Society of Ireland, XL, 13351.Young, K., Ashby, D., Boaz, A., & Grayson, L. (2002). Social science and the

evidence-based policy movement. Social Policy and Society, 1 (3), 21524.

138 FRANCES RUANE

research evidence and policy making

Documents