what science is really worth

7/28/2019 What Science is Really Worth

1/3

President Barack Obama says it. FrancisCollins, director of the US National Insti-tutes of Health (NIH), says it. Universityand research leaders elsewhere are say-

ing it, too. The number one current rationalefor extra research investment is that it will gen-erate badly needed economic growth.

Science is more essential for our prosperity,our health, our environment and our quality oflife than it has ever been before, said Obama,addressing the National Academy of Sciencesin Washington DC last year. Getting down tothe details, Collins has recently cited a report byFamilies USA, a Washington DC-based health-

advocacy group, which found that every US$1spent by the NIH typically generates $2.21 inadditional economic output within 12 months.

Biomedical research has generally beenlooked at for its health benefits, but the case forit generating economic growth is pretty com-pelling, says Collins. In Britain, senior scien-tists have called on the government to supportscience as a means of helping the economy outof recession. Heeding such arguments, govern-ments in Germany, Sweden, Canada and Aus-tralia, as well as the United States, have increasedresearch spending as part of stimulus packagesdesigned to aid their struggling economies.

Beneath the rhetoric, however, there is con-siderable unease that the economic benefits

of science spending are being oversold. TheFamilies USA study used a model developedby the Bureau of Economic Analysis at the USDepartment of Commerce to deduce the likelybenefits of NIH spending in each state. Collinssays he has been advised that the approach isstandard and considered reliable. But someeconomists question the basic assumptionbehind such models that a certain amountof research input will generate correspondingeconomic outputs or that those outputs canbe quantified.

Costs or benefits

The problem, economists say,is that the numbers attached towidely quoted economic benefitsof research have been extrapolatedfrom a small number of studies,many of which were undertaken with theexplicit aim of building support for researchinvestment, rather than being objective assess-ments. The economics of health research, onwhich much analysis of costs and benefitshas been focused, has had very little moneyinvested in it, says Martin Buxton, directorof the Health Economics Research Group atBrunel University, UK. And too much of what

has been done, has been done as a process ofadvocacy.

Research leaders acknowledge that theyneed better tools. Collins says that the NIHheld a workshop with economists in May to seewhether it should invest some of its funds intoeconomic outcomes. Were very interested intightening up the evidence base, he says.

Some of that evidence is already being col-lected. Under the programme STAR MET-RICS (Science and Technology in AmericasReinvestment Measuring the Effects ofResearch on Innovation, Competitiveness

and Science), implemented afterthe US stimulus package wasintroduced, the Obama admin-

istration is seeking to trace theeffect of federal research grantsand contracts on outcomes suchas employment, publications andeconomic activity (see Nature

464, 488489; 2010). The programmes sup-porters say it will provide justification for thestimulus money exactly what research agen-cies need as they come under pressure to showwhat recent investments have produced.

Economic arguments have always been usedto make the case for science spending, particu-larly when times are tough. In the United States,these were strengthened by the publication of

Rising Above the Gathering Storm, an influential2006 report from the US National Academies,

What science is really worthSpending on science is one of the best ways to generate jobs and economic growth, say research

advocates. But as Colin Macilwain reports, the evidence behind such claims is patchy.

Too much of

what has been

done, has beendone as a process

of advocacy.

E.R

ya

n

682

Vol 465|10 June 2010

682

NEWS FEATURE NATURE|Vol 465|10 June 2010

20 Macmillan Publishers Limited. All rights reserved10


2/3

which called for the sharp expansion of publiclyfunded research and development to stave offcompetition from China and elsewhere. The

600-page report, written by a panel chaired byNorman Augustine, former chairman of Lock-heed Martin, was put together by a large panelof senior scientists in a matter of weeks, to meeta tight congressional deadline.

In a section titled Why are science and tech-nology critical to Americas prosperity in the21st century? the report reviews the literaturethat estimates return on investment (ROI) fromresearch (see table). This is illustrated by variousgraphs, including one showing steep declines inUS death rates from heart disease between 1950and 2000, inferring that this drop can be partlyattributed to biomedical research.

Innovation driveGathering Storm recommended that federalinvestment in basic research should increaseby 10% every year for seven years,and led Congress to considerspending increases of that order,mainly in the physical sciencesand engineering. When the newlyelected President Obama was hur-riedly preparing a February 2009bill aimed at stimulating economicgrowth, parts of these increases were thrownin, together with extra spending for the NIH.

In the end, the American Recovery and Rein-vestment Act included a further $21 billion ofresearch spending, all justified by its support-ers on the grounds that it would yield speedyeconomic returns.

Yet Stephen Merrill, executive director of theBoard on Science, Technology and EconomicPolicy at the National Academies but who wasnot involved in the report, concedes that Gath-ering Storm doesnt, in itself, make a detailed

case for the economic benefits of investing inresearch. For that, one has to look further backin the literature.

Economists have agreed for decades that alarge component of modern economic growthhas to be driven by innovation that is, thearrival of new ideas and technologies. We have

very good evidence that 5070% of productiv-ity growth arises from innovation, says IainGillespie, head of the Science and Technol-ogy Policy Division at the Organisation forEconomic Co-operation and Development inParis. Greater difficulty arises in determiningwhat drives the innovation, though. Is it basicresearch, often publicly funded, as the sci-ence advocates contend? Or are other factors,such as the demands of consumers who buy,

say, mobile phones or computer games, alsoinvolved? And even if scientific research doesdrive innovation, will more investment in sci-ence necessarily speed up the process? Unfor-

tunately, economists concede, noone really knows.

In one of the bedrock papers inthis field, Edwin Mansfield, the lateUniversity of Pennsylvania econo-mist, estimated that academicresearch delivered an annual rateof return of 28% (E. Mansfield

Research Policy20,112; 1991). The figure hasbeen widely quoted ever since. But Mansfield

reached this estimate by interviewing chiefexecutives, asking them what proportion oftheir companies innovation was derived fromuniversity research and, in effect, demandingthat they come up with a number. He was ask-ing an impossible question, says Ben Martin, aformer director of the Science and TechnologyPolicy Research Unit at the University of Sus-sex, UK. Methodologically, this was a dubiousthing to do.

Whatever method economists have usedsince, measuring the ROI from research hasproved tough, and has produced a wide rangeof values (see table). Some look at the microlevel, asking things such as: what contributiondid a dozen neuroscience grants received bythe University of Cambridge in 1972 eventu-ally make to drug development? Such effortsare complicated, however, by the diff icultiesof attributing credit for any given drug tothe numerous research teams involved overtime. Policy-makers are more interested inthe macro question, measuring the effect

of combined research activities on a coun-trys economic growth. According to Merrill,repeated efforts to pin down firm numbershere have also failed. It is fair to say that this isan analytical dead end, he told attendees at theAmerican Association for the Advancement ofScience annual meeting in February.

Exceptional returns?Martin says that for much of the literature,there is some PR, rather than rigorous researchinvolved. This influence derives in part fromthe activities of US medical research lobby-ists. An example is the 2000 report Exceptional

Returns: The Economic Value of Americas Invest-ment in Medical Research by Funding First, aninitiative of the Mary Woodard Lasker Chari-table Trust that advocated biomedical researchspending. Pointing to work by various econo-mists, the document estimated that the steepdecline in cardiovascular deaths in the UnitedStates between 1970 and 1990 has an economic

value of $1.5 trillion annually, and deduced thatone-third of this $500 billion a year couldbe attributed to medical research that led to newprocedures and drugs, a finding that was ech-oed in the Gathering Storm report. A plethoraof studies in the United States and Australia fol-

lowed through with similar claims.Funding First has been disbanded, but Robert

ESTIMATES OF RATE OF RETURN ON PUBLIC RESEARCH & DEVELOPMENT INVESTMENT

Year of study Subject Annual rate of return (%)

1958 Hybrid corn 2040

1967 Poultry 2125

1979 Tomto hrvester 3746

1968 agriculturl reserch 3540





1991 all cdemic science reserch 28


2000 Phrmceuticls 30+

SOURCE: Rising Above the Gathering Storm (ntiol acdemies, 2006); Scott, G. et al.The Economic Returns of Basic Researchand the Benefits of UniversityIndustry Relationships Sciece d Techology Policy Reserch (Uiv. Sussex, 2001).

It is very hard

to take changes

in public health

and attribute

their cause.

NIH director Francis Collin is exploring new ways

to document the effect of research investment.

683

Vol 465|10 June 2010

683

NATURE|Vol 465|10 June 2010 NEWS FEATURE



3/3

Topel, who studies labour eco-nomics at the University of Chi-cago and whose work was cited

in the report, distances himselffrom some of its claims. Prob-ably only a little of the fall in thecardiovascular death rate hasto do with surgery and beta-blockers, he says. It is veryhard to take changes in publichealth and attribute their cause.Topel also questions the reportsimplication that publicly fundedbiomedical research will createthousands of jobs in the phar-maceutical and biotechnologyindustries. Topel says that Mark

Hatfield, the former Republicansenator for Oregon who wrotethe reports introduction, wasconstantly fishing for job num-bers. We kept telling Hatfield that jobs are acost, not a benefit.

The price of researchA key problem, says Topel, has been economistsinability to measure the costs of research as wellas the benefits. These costs include the addedexpense of caring for elderly patients kept aliveby new treatments, the costs of talented peopledoing research instead of something economi-

cally productive (such as running a technologycompany or an ice-cream van), and the cost ofwayward outcomes, such as nuclear clean-up a long-term outcome of the research anddevelopment of nuclear energy and weaponry.Research agencies have no interest in assessingthe costs of research fairly, says Barry Bozeman,a science-policy specialist at Georgia Instituteof Technology in Atlanta. Honest clients arein short supply for research in this field, hesays. Most of them think they already have theanswers, and want someoneto find the numbers to provethem right.

The flaws in the excep-tional returns l iteraturewere thrown into sharprelief in a November2008 study called

Medical Research:Whats it Worth?by theLondon-based WellcomeTrust and the UK MedicalResearch Council. In it, some UKhealth economists attempted to makerigorous estimates of the economicbenefits of publicly and charitablyfunded medical research in Britain.

They estimated that every poundinvested in cardiovascular disease and

mental-health research brought about, throughimproved health, economic returns of 9% and7%, respectively. Work in both fields generatedan extra return of 30% through spillover effectsfrom research to the broader economy, such astraining and industrial activity. But the reportsaid that these findings were at best tentative,and spelled out a long list of knowledge gaps.

Little is known about how long the economicbenefits of research take to accrue; nor the

extent to which the benefits of research done inone country or region are specific to that area,which is a central question for policy-makers.Three-quarters of the benefits are in spillover,and thats where the evidence is weakest, saysJonathan Grant, president of RAND Europe inCambridge, UK, and one of the studys mainauthors. Grant also questions the way that dataon ROI gathered in one sector, such as agricul-tural research, have sometimes been appliedto others. Most of the empirical evidence in

this area is (a) historical, (b) Americanand (c) from agriculture. How

transferable is that? Its

a big question in mymind, he says.Efforts to strengthen

the e v ide nce a r eincreasing. An $8-mil-lion-a-year grantsprogramme at the

National Science Foun-dation (NSF), for example, issupporting investigations by

science-policy specialists andeconomists into various aspectsof research economics, includingseveral approaches to measuring

the impact of Obamas stimuluspackage. The programme grew

out of an initiative to build ascientifically rigorous, quanti-tative basis for research policy,

launched in 2005 by John Mar-burger, then science adviserto President George W. Bush.As Marburger explains, Weneed disinterested people asopposed to the current situa-tion, where everyone involvedhas an interest in the outcome.

Julia Lane, head of the NSFproject, is also directing therelated STAR METRICS pro-gramme. The first aim of theprogramme is to build a cleandatabase of all federally funded

researchers in the United States current records are confused,with conflicting informationon names and affiliations

and estimate the number of people that theykeep in employment. Later on, the plan is totrack patents, citations and other metrics ofthe researchs impact. Lane, like Marburger,suggests that researchers use of the Internetto communicate and publish will enable STARMETRICS to track the creation and transfer ofknowledge properly for the first time. In thepast, we havent had the data infrastructure todo a full analysis, she says.

Tobin Smith, vice-president for policy at theAssociation of American Universities in Wash-ington DC, is confident that the first STARMETRICS results in summer 2011 will help toshow doubters that the stimulus-bill money hasbeen wisely spent. It will certainly help me, hesays, by telling our campuses how many peoplethey are keeping in work something universi-ties have never been able to do. Like Smith, mostresearch leaders and advocates seem assuredthat new data will reveal the healthy return oninvestment they have been touting all along.

Not that this guarantees that the economicgrowth argument will continue to persuade.

There are signs that a backlash against fur-ther research spending is already emerging.In May, the US House of Representatives deci-sively rejected a bill that would have authorizedincreased research funding for physical sciencesagencies, and in Britain, research spending cutsby the newly elected government are widelyanticipated. The pressure is building to showwhat earlier investments have produced.

As one former congressional staffer, whodidnt want to be named, puts it: If it turns outthat all the stimulus has done is hire a load offoreign postdocs, theres going to be trouble. Colin Macilwain is a freelance writer based in

Edinburgh, UK.

See Editorial, page 655.

Funding First, an advocate of research spending, attributed some economic benefits

from falling mortality rates to medical research in a 2000 report.

laSkERf

OUndaTiOn

684

Vol 465|10 June 2010

684

NATURE|Vol 465|10 June 2010NEWS FEATURE


what science is really worth

Documents