national€¦ · national science board eamon m. kelly (chair), president emeritus and professor,...

NSB 01-156

NATIONAL SCIENCE BOARD

FEDERALRESEARCHRESOURCES:A PROCESS FORSETTING PRIORITIES

NATIONAL SCIENCE FOUNDATION OCTOBER 11, 2001

iii

NATIONAL SCIENCE BOARDEamon M. Kelly (Chair), President Emeritus and Professor, Payson Center for

International Development & Technology Transfer, Tulane University

Anita K. Jones (Vice Chair), Quarles Professor of Engineering and Applied Science,Department of Computer Science, University of Virginia

John A. Armstrong, Vice President for Science & Technology, IBM (Retired)

Nina V. Fedoroff, Willaman Professor of Life Sciences, Director Life SciencesConsortium, and Director, Biotechnology Institute, The Pennsylvania StateUniversity

Pamela A. Ferguson, Professor and Former President, Grinnell College

Mary K. Gaillard, Professor of Physics, University of California-Berkeley

M.R.C. Greenwood, Chancellor, University of California-Santa Cruz

Stanley V. Jaskolski, Vice President, Eaton Corporation (Retired)

George M. Langford, Professor, Department of Biological Science, Dartmouth College

Jane Lubchenco, Wayne and Gladys Valley Professor of Marine Biology andDistinguished Professor of Zoology, Oregon State University

Joseph A. Miller, Jr., Senior Vice President and Chief Technology Officer, Corning,Inc.

Diana S. Natalicio, President, The University of Texas at El Paso

Robert C. Richardson, Vice Provost for Research and Professor of Physics,Department of Physics, Cornell University

Michael G. Rossmann, Hanley Distinguished Professor of Biological Sciences,Department of Biological Sciences, Purdue University

Vera Rubin, Research Staff, Department of Terrestrial Magnetism, CarnegieInstitution of Washington

Maxine Savitz, General Manager, Technology Partnerships, Honeywell

Luis Sequeira, J.C. Walker Professor Emeritus, Departments of Bacteriology andPlant Pathology, University of Wisconsin-Madison

Daniel Simberloff, Nancy Gore Hunger Professor of Environmental Science,Department of Ecology and Evolutionary Biology, University of Tennessee

Bob H. Suzuki, President, California State Polytechnic University

Richard Tapia, Professor, Department of Computational & Applied Mathematics, RiceUniversity

Chang-Lin Tien, University Professor and NEC Distinguished Professor ofEngineering, Department of Mechanical Engineering, University of`California-Berkeley

Warren M. Washington, Senior Scientist and Head, Climate Change ResearchSection, National Center for Atmospheric Research

John A. White, Jr., Chancellor, University of Arkansas-Fayetteville

Mark S. Wrighton, Chancellor, Washington University

Rita R. Colwell, (Member Ex Officio), Director, National Science Foundation

Marta Cehelsky, Executive Officer, National Science Board

iv FEDERAL RESEARCH RESOURCES:A PROCESS FOR SETTING PRIORITIES

NATIONAL SCIENCE BOARDAD HOC COMMITTEE ON STRATEGICSCIENCE AND ENGINEERING POLICYISSUES

The National Science Board consists of 24 members plus the Director of theNational Science Foundation. Appointed by the President, the Board serves asthe governing board of NSF and provides advice to the President and the Con-gress on matters of national science and engineering policy.

John A. Armstrong

M.R.C. Greenwood

Anita K. Jones

Joseph A. Miller

Robert C. Richardson

Warren M. Washington

Jean M. Pomeroy, Executive Secretary

Eamon M. Kelly, Chair

v

CHAPTER 2: NATIONAL POLICY CONCERNS AND NEEDS...........................13

CHAPTER 3: MAJOR FINDINGS AND ISSUES...........................................25

CHAPTER 4: CONCLUSIONS AND RECOMMENDATIONS..............................29

REFERENCES:

SELECTED ACRONYMS..................................................................39

SELECTED BIBLIOGRAPHY.............................................................41

APPENDICES:

PREFACE.....................................................................viiACKNOWLEDGMENTS.......................................................ix

EXECUTIVE SUMMARY...........................................................1

CHAPTER 1: INTRODUCTION................................................................9

CONTENTS

• Process for Producing the Report 10

• Sidebar Quotes 37• Boxed Text 38• Figures 38

A. Charge to the Committee 47B. Presentations and Discussions with Committee 49C. Agendas and Guidelines for Selected Stakeholder and Expert Meetings 51

D. Names and Affiliations of Those Submitting Written Comments 63E. Contractor Reports: Executive Summaries and Bibliographies 65

S&T Budget Coordination and Priority Setting

• May 21-22, 2001 Symposium on Allocation of Federal Resources for

Science and Technology 52• October 20, 2000 Meeting with Federal Budget Experts and

Economists• August 3-4, 2000 Meeting with Federal Agencies on the

Federal R&D Budget Allocation Process• November 18-20, 1999 Symposium on International Models for

55

56

57

• RAND, Setting Priorities and Coordinating Federal R&D Across Fields of Science: A Literature Review

• SRI International, Symposium on International Models of Budget Coordination and Priority Setting for S&T

66

99

vii

PREFACE

Science and technology have served the Nation well in war and in peace. TheFederal role in U.S. science and technology is essential and unique, enablingbroad and sustained national capabilities to seize new opportunities for scienceand technology and to respond to current and future national needs. Recentevents dramatize the importance of maintaining a future oriented science andtechnology policy to be prepared to address both the challenges we are able toarticulate now and those not yet imagined. The growing reliance on science andtechnology in government, the economy, national security and the society raisesthe urgency for adopting a comprehensive Federal process for research budgetcoordination and priority setting to align Federal support for research with broadnational goals.

In March 1999, in response to a request from the House Appropriations Com-mittee and with the encouragement of Office of Management and Budget, theNational Science Board reconstituted its Ad Hoc Committee on Strategic Scienceand Engineering Policy Issues and charged it to undertake a study on budgetcoordination and priority setting for government-funded research. This report,Federal Research Resources: A Process for Setting Priorities, presents the findingsand recommendations developed by the committee and approved unanimously bythe National Science Board. The study included review of the literature onFederal budget coordination and priority setting for research, and invited pre-sentations from and discussions with representatives of the OMB, the Office ofScience and Technology Policy, the Federal R&D agencies, congressional staff,high-level science officials from foreign governments, experts on data andmethodologies, and spokespersons from industry, the National Academies,research communities, science policy community, and academe. Discussionsfocused on research priority setting as it is practiced in government organiza-tions, and possibilities for enhancing coordination and priority setting for theFederal research budget.

On behalf of the National Science Board, I want to commend my colleagues andfellow members of the Ad Hoc Committee on Strategic Science and EngineeringPolicy Issues—Drs. John Armstrong, MRC Greenwood, Anita Jones, JosephMiller, Robert Richardson, and Warren Washington—for their tenacity, energy,and wisdom throughout this complex study. Jean Pomeroy, Senior PolicyAnalyst in the National Science Board Office provided outstanding and tirelesssupport as Executive Secretary to the Committee.

viii FEDERAL RESEARCH RESOURCES:A PROCESS FOR SETTING PRIORITIES

The Board is grateful to National Science Foundation Director, Rita Colwell, andDeputy Director, Joseph Bordogna, for their strong support and for the crucialparticipation by many members of the science and technology policy communi-ties within and outside of the Federal Government throughout this project.

Eamon M. KellyChairman

National Science Board andAd Hoc Committee on Strategic Science

and Engineering Policy Issues

ix

ACKNOWLEDGMENTS

A large number of people assisted the National Science Board’s study as speak-ers, panelists, and presenters, whose names and contributions are listed inappendices to this report. Several of our colleagues were critical to initiation ofthe Board and Committee study. These include former National Science BoardChairman Richard Zare, who oversaw the previous phase of this inquiry andFranklin Raines, former Director of the Office of Management and Budget, whochallenged the Board to address this topic.

Extensive contributions have been made over the course of this study by Officeof Management and Budget staff, Kathleen Peroff, Steven Isakowitz and DavidRadzanowski, who encouraged us and helped shape the direction of our inquiryin numerous productive conversations. Neal Lane, former Director of NSF andsubsequently of the White House Office of Science and Technology Policy andAdviser to the President for Science and Technology, deserves our specialthanks for his encouragement and crucial participation in several importantevents sponsored by the NSB Committee. Also deserving special mention areJames Duderstadt, National Acadamies’ Committee on Science, Engineering,and Public Policy and former Chairman, National Science Board, who contrib-uted to several important discussions, and encouraged the Board to pursue itsstudy; and the House Appropriations Committee Chief of Staff, Frank Cushing,who provided important insights on the Congressional perspective.

We are also grateful for the participation of many members of the science policycommunity who, like us, are wrestling with the issues addressed in this report.They include spokespersons from the National Academies; Federal departmentsand agencies; academic, industrial and professional organizations; public policyorganizations; science officials of foreign governments; and experts on data andmethodologies to guide, track and measure the impacts of Federal investmentsin research.

We appreciate the unstinting support of NSF staff. We would like to acknowl-edge the critical contributions of Richard Ries, NSF Division of InternationalPrograms, who advised on both the content and participants for the Symposiumon International Models of S&T Budget Coordination and Priority Setting co-sponsored by the Committee; Steven Payson, Science Resources StudiesDivision, who provided economic expertise; William Butts, Social, Behavioraland Economic Sciences Directorate, who coordinated SBE staff input on re-search relevant to priority setting in Federal research; and Robert Webber,Division of Administrative Services, who assisted with several special meetingsof the Committee.

The National Science Board Office staff provided essential support to theCommittee’s activities. Several NSB staff deserve special recognition, includingMarta Cehelsky, Executive Officer, National Science Board, who guided andsupported to all aspects of the Board’s effort; Daryl Chubin, Senior PolicyOfficer, NSB, who provided helpful feedback and guidance and Gerard Glaser,his successor; and Janice Baker, NSB Writer/Editor.

1

EXECUTIVE SUMMARY

CONTEXT AND FRAMEWORK FOR STUDY

The Federal Government’s policy for investment in science and technology overthe last 50 years has yielded enormous benefits to the economy, nationalsecurity, and quality of life in the U.S. The Federal share of total nationalscience and technology investment is critically focused in areas that would beinadequately funded or not supported by the private sector. These includeresearch to support Federal missions; research that is high-risk or requireslong-term investment in the expectation of future high payoffs to society;unique, costly, cutting-edge research facilities and instruments; and academicresearch that, as a primary purpose, supports the education of the futurescience and engineering workforce.

Over $90 billion1 was allocated to Federal R&D in the most recent budget—representing a little more than a quarter of all national R&D. With such alarge investment of public funds, policy makers in Congress and the Executivebranch are asking for convincing evidence of the effectiveness of Federalinvestments in the form of hard data on benefits. There is general recognitionamong policy makers that outstanding opportunities for excellent research farexceed any reasonable level of funding by the Federal government. Choicesmust be made. Wise, well-informed choices among alternatives will sustain astrong, balanced research infrastructure to enable the discoveries that will be afoundation for future prosperity.

The current system for priority setting in the Federal research budget lacks acoherent, scientifically based process for systematic review and evaluation ofthe broad Federal investment portfolio for effectiveness in achieving nationalgoals. Moreover available data and analyses are often ill suited for informing

“Our challenge, now andin the future, will be tomaintain a steady flow ofunderstanding-drivenscientific and engineeringstudies even in the face oflimited federal resources.Meeting this challenge meansthat priorities for spending onscience and engineering bythe federal government willhave to be set.” — U. S. Houseof Representatives, UnlockingOur Future

1 Executive Office of the President, Office of Management and Budget, “Analytical Perspectives,” FiscalYear 2002 Budget of the United States Government, Table 7-2, Research and Development Spending.

budget allocation decisions that affect U.S. research infrastructure.

2 FEDERAL RESEARCH RESOURCES:A PROCESS FOR SETTING PRIORITIES

“Science and technologyare critically important tokeeping our nation’seconomy competitive andfor addressing challengeswe face in health care,defense, energy productionand use, and theenvironment. As a result,every federal research anddevelopment (R&D) dollarmust be invested aseffectively as possible.” —OMB, The President’sManagement Agenda,Fiscal Year 2002

Decision makers must rely on the scientific community to provide the bestadvice on the most promising research investment choices for the future. Theform and timing of such advice are also important. Appropriate advice mustinclude a reasonable estimate of the level of funding that would be required foradequate support of a new initiative over time, provide tradeoff options toenable funding for priorities, and be available on a schedule compatible with theFederal budget process.

No process now exists for weighing the available evidence on competing re-search investment opportunities across broad fields of research. It is criticalthat the choices among such opportunities be based on a process that istransparent and credible with the scientific communities and the general publicand its representatives. Such a function requires an organizational home,appropriate expert resources, and adequate financial support.

Since the mid 1990s, the National Science Board has been actively engaged inissues of priority setting for the Federal research portfolio.2 In 1999, the Boardcharged its Ad Hoc Committee on Strategic Science and Engineering PolicyIssues to undertake a study of research budget coordination and priority settingmethodologies across fields of science and engineering in the U.S. and in othercountries.

CONDUCT OF STUDY

The study, Federal Research Resources: A Process for Setting Priorities (NSB 01-156), which follows on recommendations of the Board’s previous working paperon Government Funding of Scientific Research [NSB 97-186), responds to a requestby the House Appropriations Committee3 and the encouragement of the Office ofManagement and Budget. In its November 1998 Strategic Plan (NSB-98-215) theBoard identified this effort as a high priority for national science policy.

The Committee on Strategic Science and Engineering Policy Issues commis-sioned reviews of the literature in two areas.4 The first focused on Federalresearch budget coordination, priority setting across fields of science andengineering, and available data and analytical tools to support priority setting.A second study of the same subject reviewed international models of S&Tbudget coordination and priority setting. It also included a symposium withpresentations by S&T officials from eight foreign governments.

2 The National Science Board issued In Support of Basic Research—NSB 93-127; Federal Investments inScience and Engineering—NSB 95-254 and Statement on Federal R&D Budget Realignment—NSB 95-26,from 1993 to 1995, in addition to more recent papers.3 House Appropriations Committee Report 105-610, 105th Congress, 2nd Session, U. S. House ofRepresentatives. To accompany H.R. 4194.4 Steven W. Popper, Caroline S. Wagner, Donna L. Fossum, William S. Stiles. Setting Priorities andCoordinating Federal R&D Across Fields of Science: A Literature Review (DRU-2286-NSF). Washington DC:RAND Science and Technology Policy Institute, April 2000; and H. Roberts Coward. Final Report: Sympo-sium on International Models of Budget Coordination and Priority Setting for S&T. Washington DC: SRIInternational, August 2000.

3

In addition to these studies, the Committee heard presentations by invitedexperts who discussed a wide range of methodologies and data to supportbudget allocation decisions for research. It also received written comments onits draft recommendations by mail and through the National Science Boardwebsite, and heard presentations broadly representative of stakeholders inFederal research. Stakeholder input culminated with a Symposium on May 21-22, 2001 on the Board’s preliminary findings and recommendations, with morethan 200 participants.

EXECUTIVE SUMMARY

PRINCIPAL FINDINGS

Federal priority setting for research occurs at three levels: 1) establishingFederal goals for research, 2) the budget allocation processes for researchwithin the White House and Congress that in the aggregate produce theFederal research portfolio and 3) Federal agencies and departments inachieving their missions in accord with the President’s priorities forresearch. This report focuses on the second level, that is, the WhiteHouse and Congressional processes that in the aggregate produce theFederal portfolio of investments in research.

The allocation of funds to national research goals is ultimately a politicalprocess that should be informed by the best scientific advice and data avail-able.

A strengthened process for research allocation decisions is needed. Suchallocations are based now primarily on faith in future payoffs justified bypast success. They are difficult to defend against alternative claims on thebudget that promise concrete, more easily measured results and aresupported by large and vocal constituencies.

The pluralistic framework for Federal research is a positive aspect of thesystem and increases possibilities for funding high-risk, high-payoff research.An improved process for budget coordination and priority setting should buildon strengths of the current system and address weaknesses in data, analy-ses, and expert advice.

There is a need for regular evaluation of Federal investments as a portfoliofor success in achieving Federal goals for research, to identify areas of weak-ness in national infrastructure for S&T, and to identify a well-defined set ofthe top priorities for major new research investments.

Additional resources are needed to provide both Congress and the Execu-tive branch with data, analyses, and expert advice to inform their decisionson budget allocations for research.


RECOMMENDATIONS

Implementation of a broad-based, continuous capability for expert advice toboth OMB and Congress during the budget process would yield immediatebenefits to decision makers. There is also a long-term need for a regular,systematic evaluation of the effectiveness of Federal investments in achievingFederal goals for research through the Office of Science and Technology Policy,drawing broad-based input from scientific experts and organizations in allsectors. Complementing both would be improved analyses on research opportu-nities, needs, and benefits to society; and timely data that trace Federal re-search investments through the budget process and beyond.

KEYSTONE RECOMMENDATION 1

The Federal Government, including the White House, Federal depart-ments and agencies, and the Congress should cooperate in developingand supporting a more productive process for allocating and coordi-nating Federal research funding. The process must place a priority oninvestments in areas that advance important national goals, identifyareas ready to benefit from greater investment, address long-termneeds and opportunities for Federal missions and responsibilities,and ensure world class fundamental science and engineering capabili-ties across the frontiers of knowledge. It should incorporate inputfrom the Federal departments and agencies, advisory mechanisms ofthe National Academies, scientific community organizations repre-senting all sectors, and a global perspective on opportunities andneeds for U.S. science and technology.

RESEARCH COMMUNITY INPUT ON NEEDS ANDOPPORTUNITIES:

Presently there is no widely accepted and broadly applied way for the FederalGovernment to obtain systematic input from the science and engineering commu-nities to inform budget choices on support for research and research infrastruc-ture. The current system often fails to produce advice and information on aschedule useful to the budget process and responsive to needs for broad-based,informed assessments of the benefits and costs of alternative proposals forFederal support. A more effective system for managing the Federal researchportfolio requires adequate funding, staffing and organizational continuity.

5EXECUTIVE SUMMARY

RECOMMENDATION 2

A process should be implemented that identifies priority needs andopportunities for research—encompassing all major areas of scienceand engineering—to inform Federal budget decisions. The processshould include an evaluation of the current Federal portfolio forresearch in light of national goals, and draw on: systematic, indepen-dent expert advice from the external scientific communities; studiesof the costs and benefits of research investments; and analyses ofavailable data; and should include S&T priorities, advice, and analysesfrom Federal departments and agencies. The priorities identifiedwould inform OMB in developing its guidance to Federal departmentsand agencies for the President’s budget submission, and the Congressin the budget development and appropriations processes.

EXECUTIVE BRANCH ADVISORY MECHANISM:

The Executive Branch should implement a more robust advisory mechanism,expanding on and enhancing current White House mechanisms for S&T budgetcoordination and priority setting in OSTP and OMB. It is particularly essentialthat the advisory mechanism include participants who are experienced in makingchoices among excellent opportunities or needs for research, for example, viceprovosts for research in universities, active researchers with breadth of vision,and managers of major industrial research programs.

RECOMMENDATION 2A

An Executive Branch process for ongoing evaluation of outcomes ofthe Federal portfolio for research in light of Federal goals for S&Tshould be implemented on a five-year cycle.5 A report to the Presi-dent and Congress should be prepared including a well-defined set ofthe highest long-term priorities for Federal research investments.These priorities should include new national initiatives, unique andparadigm shifting instrumentation and facilities, unintended andunanticipated shifts in support among areas of research resulting ingaps in support to important research domains, and emerging fields.The report should also include potential trade-offs to provide greaterfunding for priority activities. The report should be updated on anannual basis as part of the budget process, and should employ the bestavailable data and analyses as well as expert input. Resources avail-able to OSTP, OMB and PCAST should be bolstered to support thisfunction.

5 The designation of a five-year cycle for evaluation of the Federal portfolio reflects both the scale of theeffort, which would require a longer time than an annual process, and the increasingly rapid changes inscience that demand a frequent reevaluation of needs and opportunities for investments.


CONGRESSIONAL ADVISORY MECHANISM:

There is no coherent congressional mechanism for considering allocationdecisions for research within the framework of the broad Federal researchportfolio. Though improvements in the White House process—particularlyexpansion of activities and resources available to OSTP—would benefit congres-sional allocation decisions, one or more congressional mechanisms to provideexpert input to research allocation decisions are badly needed.

RECOMMENDATION 2B

Congress should develop appropriate mechanisms to provide it withindependent expert S&T review, evaluation, and advice. Thesemechanisms should build on existing resources for budget andscientific analysis, such as the Congressional Budget Office, theCongressional Research Service, the Government Accounting Office,and the National Academies. A framework for considering the fullFederal portfolio for science and technology might include hearingsby the Budget Committees of both houses of Congress, or other suchbroadly based congressional forums.

DEFINITIONS, DATA AND DATA SYSTEMS:

High quality data and data systems to monitor Federal investments in researchwould enhance the decision process. Such systems must be based on defini-tions of research activities that are consistently applied across departmentsand agencies and measured to capture the changing character of research andresearch needs. Improving data will require long-term commitment with inputfrom potential users and contributors, and appropriate financial support.

RECOMMENDATION 3

A strategy for addressing data needs should be developed. Such astrategy supported by OMB and Congress and managed through OSTPand OMB would assure commitment by departments, agencies andprograms to timely, accessible data that are reliable across reportingunits and relevant to the needs for monitoring and evaluatingFederal investments in research. Current data and data systemstracking federally funded research should be evaluated for utility tothe research budget allocation process and employed as appropriate.

7EXECUTIVE SUMMARY

INTERNATIONAL COMPARISONS:

Both relative and absolute international statistical data and assessmentsshould be a major component of the information base to support ExecutiveBranch and Congressional research budget allocation decisions. Internationalbenchmarking of U.S. research performance and capabilities on a regular basisresponds to the growing globalization of science and technology and the need forthe U.S. to maintain a world-class science and engineering infrastructure.

RECOMMENDATION 4

Input to Federal allocation decisions should include comparisons ofU.S. research resources and performance with those of other coun-tries. National resources and performance should be benchmarkedto evaluate the health and vigor of U.S. science and engineering for arange of macroeconomic indicators, using both absolute and relativemeasures, the latter to control in part for the difference in size andcomposition of economies. Over the long term, data sources shouldbe expanded and quality improved.

FEDERAL RESEARCH BENEFITS TO THE ECONOMY ANDSOCIETY:

In addition to monitoring Federal expenditures for research, measuring thebenefits to the public of funded research is essential for prudent management.Implementation of this recommendation should be coordinated with Recommen-dation 3 on definitions and data systems.

RECOMMENDATION 5

The Federal Government should invest in the research necessary tobuild deep understanding and the intellectual infrastructure toanalyze substantive effects on the economy and quality of life ofFederal support for science and technology. The research shouldinclude improvements to methods for measuring returns on publicinvestments in research.


CONCLUSION

The Board’s recommendations provide a framework for improving the quality,content, and accessibility of science and engineering expert advice, data, andanalyses to inform decisions on priorities in the White House and Congress forFederal investments across fields of research. We are aware that implementingthese recommendations will be difficult and require long-term commitment andsupport. In the interest of science and the Nation, we urge that the FederalGovernment and its partners in the research community embrace this difficulttask.

9

INTRODUCTION

In the fifty years since the end of World War II and the establishment of anational policy for Government support of scientific research in colleges anduniversities,6 historical trends and events have changed the public expectationsfor Federal research investments. The most important historical event affectingthe national post-World War II consensus on Federal participation in scienceand technology is the end of the Cold War. Until that time, the rationale forFederal investments in research relied heavily on the contributions of scienceand technology (S&T) to a strong national defense.

The last few Federal budget years have been favorable to research, but a favor-able budget in one or two fiscal years does not obviate the need for a coherentpost-Cold War Federal policy and decision process to guide investment in S&T.It is difficult to envision a reversal of the tide of accelerating competition amongexploding scientific opportunities and between science and other worthy claim-ants on the budget. Today’s environment demands more effective managementof the Federal portfolio for research, including a sustained advisory process thatincorporates systematic involvement of participants in the U.S. research enter-prise, including the science and engineering communities, Federal agencies thatfund research, industry, nonprofit organizations that fund and perform research,and, increasingly, state governments. Expert input is particularly important fordecisions on long-term, high-risk investments in research–-sponsored mainly bythe Federal Government-–which are steadily losing ground in the nationalresearch portfolio to short-term investments.

The Federal commitment to research over the last half-century has contributedto a continuous outpouring of benefits to the public from advances in scienceand technology. Furthermore, within the last few decades these benefits havebecome increasingly visible and pervasive, from economic growth driven by hightechnology industries, to science and technology based transformations in manyareas of public and private life—including, among others, the revolution incommunications and information technologies, major medical breakthroughs,and superior defense technology demonstrated in the field. These transforma-

CHAPTER ONE

“Yet, in holding scientificresearch and discovery inrespect, as we should, wemust also be alert to the equaland opposite danger thatpublic policy could itselfbecome the captive of ascientific technological elite.It is the task of statesmanship tomold, to balance, and tointegrate these and otherforces, new and old, withinthe principles of ourdemocratic system—everaiming toward the supremegoals of our free society.”—President Dwight D.Eisenhower, Farewell Address,1/17/61

6 Vannevar Bush, Science—The Endless Frontier (National Science Foundation 40th Anniversary edition)Washington DC: National Science Foundation, 1990.


PROCESS FOR PRODUCING THE REPORTThe study responds to the House Committee on Appropriations FY 1999 report urging the Board to undertakethe study and the encouragement from Office of Management and Budget (OMB) for this effort. The Board AdHoc Committee on Strategic Science and Engineering Policy Issues was charged to examine the state of theart in budget coordination and priority setting for research across fields of science and engineering in theU.S. and internationally, and to convene appropriate stakeholders to consider the findings of these studiesand reviews, to develop recommendations for improved methodologies for coordination and priority setting inthe Federal research budget and for building the support of the science and engineering communities and thepublic for these methodologies. The study included two literature reviews:

Federal research budget coordination and priority setting International models of science and technology budget coordination and priority setting.

The Committee heard presentations by invited experts addressing the following methodologies andtopics:

International models for S&T budget coordination and priority setting; a one-and-a-half daySymposium was held in November, 1999, opened by the U.S. President’s Science Advisor, withpresentations by officials for eight foreign governments: the United Kingdom, Germany, France,Sweden, the Republic of Korea, Japan, Brazil, and the European Union;

A project to develop a more complete and accessible database for tracking Federal R&D funding, theRaDiUS database, and data issues in tracking S&T activities in the Federal budget;

Foresight methods, used by many countries as part of the dialogue toward establishing priorities forS&T;

The Federal Science and Technology (FS&T) budget analysis by the Committee on Science, Engineeringand Public Policy (COSEPUP) of the National Academies and the American Association for theAdvancement of Science;

Experiments in international benchmarking of U.S. research fields—undertaken by COSEPUP; Approaches to priority setting for research in the academic sector, and the relationship between

Federal and academic priority setting; Priority setting practices in industry, and the role of industry and the Federal government in national

R&D; Economic methods to measure the benefits of Federal investments in research and to inform budget

allocation decisions, presented by academic experts on economic methods to measure returns onresearch investments and experts on the Federal budget from the Congressional Budget Officeand Council of Economic Advisors.

The study included an overview of budget coordination and priority setting in Federal S&T agencies in aAugust 3-4, 2000 meeting with representatives of the Office of Science and Technology Policy (OSTP), Office ofManagement and Budget (OMB), Department of Energy (DOE), National Institutes of Health (NIH), NationalScience Foundation (NSF), National Aeronautics and Space Administration (NASA), U.S. Department ofAgriculture (USDA), Environmental Protection Agency (EPA), National Oceanographic and Atmospheric Admin-istration (NOAA), Department of Defense (DOD), Veterans Health Administration (VHA), National Instituteof Standards and Technology (NIST), and a discussion with House Appropriations staff.

Finally, the Committee sponsored a stakeholders’ symposium on Allocation of Federal Resources for Scienceand Technology, May 21-22, 2001, focused on the Board-approved discussion document, The Scientific Alloca-tion of Scientific Resources (NSB 01-039), containing preliminary recommendations. The report and invitationto the symposium were distributed by webpage, email and mail to members of the stakeholder communities.The symposium included 20 speakers and panelists and encouraged active audience participation. It wasattended by more than 200 members of the stakeholder communities including representatives from Congres-sional staff, science policy organizations, Federal agencies, academic organizations, scientific communityorganizations, science media, industry representatives and interested individuals.

11

1985 1990 1995 20001980197519701950 1955 1960 19650

.5

1.0

1.5

2.0

2.5

3.0

3.5

Percent

Since the 1960s, national expenditures for R&D have been in the range of 2 to 3percent of GDP

FIGURE 1:US R&D/GDP: 1953-2000

tions underscore the value of sustained public investments reaching backdecades. Moreover, even as the Federal share of funding has declined in na-tional research and development (R&D), non-Federal sectors of the economy—industry, academe, state and non-profit—have come to rely on the FederalGovernment to play a critical role in funding long-term investments in scienceand engineering discovery, education and innovation.

The success of the U.S. in encouraging the growth of its high technology indus-trial sector through public funding for science and engineering research andadvanced education led to the U.S. system becoming a widely emulated interna-tional model. As Federal Reserve Chairman Alan Greenspan7 noted: “ . . . theresearch facilities of our universities are envied throughout the world . . . Thepayoffs in terms of the flow of expertise, new products, and start-up companies,have been impressive.” Nonetheless, recognition of the benefits of past publicinvestments does not guarantee public support of the science and technologyinfrastructure necessary to enable future discoveries that may not yield mea-surable benefits for decades. Critics and supporters alike note the need for aclearly articulated and compelling rationale for Federal investments in scienceand technology equivalent in persuasive powers to the rationale of the ColdWar.

7 Remarks by Chairman Alan Greenspan, “Structural change in the new economy,” before the NationalGovernors’ Association, 92nd Annual Meeting, State College, Pennsylvania, July 11, 2000.

13NATIONAL POLICY CONCERNS AND NEEDS

8 Executive Office of the President, Office of Management and Budget, Fiscal Year 2002 Budget of theUnited States Government, Table 7-2, Special Analyses.9 Franklin D. Raines. “Making the Case for Federal Support of R&D,” Science (12 June 1998):1671.

1985 1990 1995 20001980197519701950 1955 1960 19650

10

20

30

40

50

60

70

80

Percent

FEDERAL

INDUSTRY

OTHER

NATIONAL POLICY CONCERNSAND NEEDS

Decision makers in the Executive and Legislative branches of government areconcerned about the management of Federal investments in research, which inthe most recent budget had reached more than $90 billion for R&D.8 Articulat-ing this concern, former Office of Management and Budget (OMB) DirectorFranklin Raines9 raised the following questions: How large a scientific enter-prise does the United States need? How can we set priorities in the Nation’sR&D enterprise? How can we measure the success of our Nation’s researchprograms? How can we strengthen the government-university partnership? Howdo we engage the American people in the excitement and wonder of science?

CHAPTER TWO

Since 1980 Industry has become increasingly dominant in national R&D.

FIGURE 2:Shares of National R&D Funding by Source: 1953-2000


Likewise, Chairman of the House Science Committee, Sherwood Boehlert,10

responded to the proposal to double Federal funding for research by questioning:“What are we going to get for that money? How will we know if we are under- orover-spending in any field? . . . We really need to push for more data.” Hewent on to warn: “I want the Committee, early on, to take a serious look at thebalance within the federal research portfolio . . . You can . . . count on me toask tough and uncomfortable questions to ensure that the scientific communityis acting in its and the nation’s long term interest.”

How should the scientific community respond to these questions and expecta-tions? How should it raise public awareness that the quality of life in the futurewill depend in large measure on the generation of new wealth, on safeguardinghuman health and the health of our planet, and on opportunities for enlighten-ment and individual development made possible by science and engineeringdiscoveries? Will the response of the scientific community be effective againstcompeting claims on the Federal budget?

These issues have prompted a vigorous policy debate over the last decadeinvolving the Executive and the Legislature, the National Academies, andprofessional societies. Nonetheless, the debate to this point has generated nowidely accepted process for the Federal Government, with systematic input fromthe scientific community and its representatives in all sectors of the economy,to make priority decisions about the allocations in and across fields of researchin support of Federal goals.

The National Science Board (the Board) has participated in this debate, issuinga series of policy statements, including an NSB working paper on GovernmentFunding of Scientific Research in 1997. The Board concluded in that paper thatwithin the Federal budget there should be an overall strategy for research, withareas of increased and decreased emphasis and a level of funding adequate bothto serve national priorities and to foster a world-class scientific and technicalenterprise. To this end Congress and the Administration need to establish aprocess that examines the Federal research budget before the total Federalbudget is disaggregated for consideration by congressional committees. TheBoard further concluded in its 1998 Strategic Plan that a prerequisite for acoherent and comprehensive Federal allocation process for research is thedevelopment of an intellectually well-founded and broadly accepted methodologyfor setting priorities across fields of science and engineering.

As follow-up to its earlier work, the Board undertook, beginning in March 1999, afocused examination of Federal priority-setting methodologies for research inthe United States at three levels: 1) setting Federal goals, 2) allocation deci-sions by Congress and the Administration that produce the Federal portfolio forresearch and 3) Federal agencies and departments in achieving their missionsin alignment with stated Federal priorities. The Board determined that theappropriate level for its focus is the second level, that is, the White House andcongressional processes that in the aggregate produce the Federal portfolio ofinvestments in research.

“I doubt that anyonewould sign on to a researchproject as poorly designedas our current nationalexperiments in science andtechnology policy . . . ourscientific enterprise remainsadrift, without a connectionto the broader society” –Representative George E.Brown, Jr., 1998

10 Congressman Sherwood Boehlert (R-NY), Speech To Universities Research Association, January 31,2001.

15NATIONAL POLICY CONCERNS AND NEEDS

The national science and technology enterprise has grown and become morepervasive in both the private sector and in government, even as the Federalshare of support to the enterprise has declined. Now, more than ever, achiev-ing Federal goals for sustaining U.S. leadership in S&T demands partnershipsand cooperation with other sectors. Understanding where Federal funding canbe best employed and the level of investment required to assure the health ofU.S. science and technology are essential to prudent management of theFederal portfolio. Commitment to an intellectually well-founded, long-termstrategy for Federal research must be an integral aspect of a sound fiscal policy,regardless of year-to-year fluctuations in available funds. The Federal budgetprocess for research must assure sustained and sufficient support for adiverse, flexible, opportunistic portfolio of investments, emphasizing the long-term health of the knowledge base and infrastructure for research—includinghuman resources.

NEED FOR A DIFFERENT APPROACH TO BUDGETCOORDINATION AND PRIORITY SETTING

The Board’s discussions with spokespersons from Executive and Legislativebranches and with experts on the budget, data and analytic methods, as well asreviews of the literature on budget coordination and priority setting, identifiedthe following needs.

Methodologies for: Determining the appropriate size of the enterprise Determining the appropriate level of support to individual fields Achieving balance in the portfolio Setting priorities for the Nation’s research enterprise Achieving effective communication on scientific matters with the

American people Strengthening government partnerships and collaboration in research

with other sectors and other international partners

CONTEXT FOR FEDERALLY FUNDED RESEARCH

The Federal role has always encompassed the missions of Federal agencies anddepartments and, beyond those missions, has helped to sustain a healthynational infrastructure for S&T. The Federal role today is especially critical forresearch that is high risk, requires long-term investment in the expectation offuture high payoffs to society or that is unlikely to be funded by the privatesector; for unique, costly, cutting-edge research facilities and instrumentation;and for academic research that, as a primary purpose, supports the education ofthe future science and engineering workforce. It is this portion of the Federalresearch portfolio that is especially vulnerable to diversion of funds to areas ofresearch with more clear and immediate payoffs to society or to other importantgoals of Federal mission departments and agencies. It is also the criticalFederal investment in our Nation’s future science and technology capabilities.


Improved data, expert analyses, and scientific advice include: A continuing mechanism for expert advice representing a broad cross-

section of the science and engineering research and education community to support difficult decisions on research investments— especially in major infrastructure projects Better quantitative data and methods of analysis adequate to measure

the benefits of research A mechanism to identify and track the relevant Federal funds for S&T

through the budget process in the Administration and Congress

THE CURRENT FEDERAL SYSTEM

The current Federal system for allocating funds for research is an incrementalprocess that results in final allocation decisions based on input from a range ofstakeholders, including the science and engineering communities. Ultimately,the Federal budget for research rests on aggregated political decisions inthirteen congressional appropriations subcommittees. There has been a host ofcritiques and suggestions for improving the process, many focused on the goalsfor research, but some suggesting changes to the process itself. The mostfrequent critique addresses a perceived lack of a clear methodology for prioritysetting and coordination. Several possible remedies have been suggested:structural changes to the process, alternative interpretations of the appropriategoals for Federal research, and new mechanisms for funding allocations andbetter management of the Federal research portfolio.

Since the late 1980s, and under both Republican and Democratic admin-istrations, there has been substantial attention devoted to developing bettermechanisms for coordinating the Federal budget for research through OMB andthe Office of Science and Technology Policy (OSTP) (Box One).

17

Office of Science and Technology Policy (OSTP): The legislation thatestablished OSTP “Declares that the United States shall adhere to anational policy for science and technology which includes the followingprinciples: (1) the continuing development and implementation of anational strategy for determining and achieving the appropriate scope,level, direction, and extent of scientific and technological efforts basedupon a continuous appraisal of the role of science and technology inachieving goals and formulating policies of the United States; (2) theenlistment of science and technology to foster a healthy economy inwhich the directions of growth and innovation are compatible with theprudent and frugal use of resources and with the preservation of a benignenvironment; and (3) the development and maintenance of a solid basefor science and technology in the United States.”It “states the declaration of Congress that the Federal Governmentshould maintain central policy-planning elements in the executive branchin mobilizing resources for essential science and technology programs, insecuring appropriate funding for those programs, and to reviewsystematically Federal science policy and programs and to recommendlegislative amendments when needed”. The functions of the Officeinclude: “(1) advise the President of scientific and technologicalconsiderations involved in areas of national concern; (2) evaluate thescale, quality, and effectiveness of the Federal effort in science andtechnology and advise on appropriate actions; (3) advise the President onscientific and technological considerations with regard to Federal budgets;and (4) assist the President in providing general leadership andcoordination of the research and development programs of the FederalGovernment.” (Excerpted from Public Law 94-292)

National Science And Technology Council (NSTC): The NSTCfunctions were to:

1) coordinate the science and technology policy-making process;2) ensure science and technology policy decisions and programs are consis-

tent with the President’s stated goals;3) help integrate the President’s S&T policy agenda across the Federal

Government;4) ensure S&T are considered in development and implementation of Federal

policies and programs; and5) further international cooperation in science and technology

(Executive Order 12881, November 23, 1993, Section 4).

Federal Coordinating Council for Science, Engineering andTechnology (FCCSET): Established in 1976 under OSTP in the NationalScience and Technology Policy, Organization, and Priorities Act, FCCSETwas “to consider problems and developments in fields of science,engineering, and technology and related activities affecting more than oneFederal agency, and to recommend policies designed to provide moreeffective planning and administration of Federal scientific, engineering,and technological programs.” (Title IV, Public Law 94-292).

WHITE HOUSE S&T POLICY APPARATUSBox One

NATIONAL POLICY CONCERNS AND NEEDS


The cabinet-level National Science and Technology Council (NSTC) (Box One) inthe previous Administration and the earlier Federal Coordinating Council forScience, Engineering and Technology (FCCSET) provided mechanisms in OSTP foridentifying major national initiatives that cut across agencies in designatedpriority areas (e.g., nanotechnology, global climate change, and informationtechnology). Under the last Administration, the NSTC was established byExecutive order as part of the OSTP science and technology policy apparatus.However, unlike FCCSET, OSTP and the Director of OSTP, which were establishedthrough legislation, the NSTC had no permanent status. Likewise, thePresident’s Committee of Advisors on Science and Technology (PCAST), whosepurpose was to provide “critical links to industry and academia,” was establishedby Executive order.

Furthermore, in neither the Executive nor the Legislative branches is there amechanism for evaluation that takes into account the breadth of Federalinvestments within the context of Federal goals for research. The ExecutiveBranch, through OMB, OSTP and PCAST, made an effort to treat Federal fundingof research as a portfolio, recently taking into account the issue of balance amongfields of science in Federal support across all agencies and departments.

These steps have been in the right direction but are only a preliminary effort.Congress also has directed attention to what might be done to improve itsprocess but has not yet taken any action to implement formal mechanismscomparable to OSTP to coordinate functions across budget lines, agencies anddepartments, and committees.

BUDGET COORDINATION AND EVALUATION OF THEPORTFOLIO

To enhance the effectiveness of Federal investments in achieving long-term goalsfor research, a regular, credible process that relies in part on expert input fromthe science and engineering communities is essential for priority setting amongcompeting investment choices. The Federal portfolio for research is an aggregateof the research portfolios of the individual departments and agencies fundingS&T. It has not been managed as a portfolio. As a precondition for prioritysetting across the Federal research budget, coordination must be achieved amongits diverse components. While efforts at better coordination through OSTPmechanisms have been useful in managing cross-agency initiatives, coordinatingmechanisms are also necessary for evaluating the performance of Federalresearch investments as a portfolio and for identifying gaps, overlaps, areas fordecreased emphasis, and the top priorities for additional investments.Coordination and priority setting therefore must be intertwined in the Federalresearch budget process.

“The Federal role today isespecially critical forresearch that is high risk,requires long-terminvestment in the expec-tation of future highpayoffs to society or that isunlikely to be funded bythe private sector; forunique, costly, cuttingedge research facilities andinstrumentation; and foracademic research that, asa primary purpose,supports the education ofthe future science andengineering workforce.”

19

THE NEED FOR MORE AND CREDIBLE DATA ANDANALYSES

No mechanism exists to provide strong quantitative input to justify a particularlevel of investment in Federal research based on expected benefits to society,due in part to the lack of data and methods to measure research benefits. Dataon Federal research funding, especially at the field level, are often unavailableon a timely basis to inform budget allocation decisions, use outdated researchfield definitions, fail to capture important characteristics of researchactivities—particularly growing collaboration across fields, organizations,sectors, and even nations—and suffer from inconsistent applications ofdefinitions across reporting units.

In spite of the need for more and better data on the Federal researchenterprise, collecting such data requires consistent cooperation of a largenumber of Federal agencies and departments. There are few resources availableto address the major undertaking that would be required to generate reliabledata tailored to the needs of budget decisions and outcomes for researchfunding allocations.

It would require a concerted effort to define and obtain agreement among themany Federal units that would be involved, and would require support from OMBand Congress to assure collection of high quality, timely data tailored totracking the Federal funding for science and technology through the budgetprocess and beyond. Nonetheless, National Science Foundation and othermajor research funding agencies have been open to developing consistent and

COORDINATING THE BUDGET FOR S&T IN CONGRESS

At no time in the congressional authorization or appropriations process isthe research portfolio examined as a whole, across the Federal govern-ment. The consideration of segments of the research budget in a largenumber of committees and subcommittees makes it impossible for Con-gress to consider the impacts of individual funding decisions on U.S.science and technology capabilities. The House Science Policy Study,Unlocking Our Future, argues that:

. . . at a minimum Congress and the Executive Branch should improve theirinternal coordination processes to more effectively manage, execute, and inte-grate oversight . . . While the Office of Management and Budget can fill this rolein the Executive Branch, no such mechanism exists in the Congress. In thosecases where two or more Congressional committees have joint jurisdiction overor significant interest in large, complex technical programs, the affected commit-tees should take steps to better coordinate their efforts. Wherever possible, theaffected committees should consider holding joint hearings and perhaps evenwriting joint authorization bills.

BOX TWO



“The nation must reach acommon ground and define amore realistic, pragmaticframework for allocatingfederal R&D resources. Onlyan inclusive national dialoguethat brings together both theexecutive and legislativebranches of government withthe private sector and the U.S.university community willproduce the neededconsensus.”—Council on Competitiveness,Endless Frontier, LimitedResources, 1996

There have been several attempts to provide a better measure for the federallyfunded activities that contribute to national innovation. The National Acad-emies proposed a coordinated “Federal Science and Technology Budget”,11 asubset of Federal R&D that constitutes “federal support for a national scienceand technology base.” The FS&T budget would provide Congress with a tool fortracking the aggregated pool of Federal departmental and agency funds thatsupport the science and technology base. OMB has employed over the last fewbudget cycles12 a similar mechanism for tracking the President’s researchpriorities through the budget process. This mechanism comprised a collectionof program budgets that are primarily research programs but also includes non-research elements, such as the education and human resources component ofthe budget for NSF. OMB found this mechanism useful in highlighting Federalresearch investments and effective in supporting the President’s priorities forresearch through the budget cycle.

The Board, for the purposes of this study, has focused on S&T. In so doing, theBoard follows the approach of organizations such as the National ResearchCouncil and OMB, which identify basic and applied research activities fortracking through the budget process. At the same time the Board recognizesthat S&T has been defined in a variety of ways in the Federal portfolio, and thatas yet there is no consensus on federally funded activities that shouldconstitute Federal S&T. Criteria for inclusion of activities in a Federal budgetfor research for the purpose of monitoring and evaluating Federal activities as aportfolio will require further discussion and analysis.

The important subset of research funding devoted to the long-term, high-riskbasic research is especially vulnerable to becoming invisible in the largerbudget for S&T. It is critical that this component receive sustained publicsupport to produce as yet unforeseen major breakthroughs in knowledge and,when performed in academic institutions, to provide opportunities forexperience in cutting-edge research for advanced science and engineeringstudents under the guidance of faculty mentors.

11 National Research Council, Committee on Criteria for Federal Support of Research and Development,Allocating Federal Funds for Science and Technology.12 Office of Management and Budget, Fiscal Year 2002 Budget, Table 7-3.

appropriate data tools for managing the Federal research portfolio and forcommunicating with more credibility to the public concerning their investmentsin research and education.

IDENTIFYING THE COMPOSITION OF THE FEDERAL RESEARCH PORTFOLIO

OMB requires agencies to report R&D activities that they are funding for theannual budget process. Even if reliably measured across funding units, since“D” (development) at about 55 percent of the total is larger than “R” (research),reporting the sum of the two as the measure of Federal research investmentresults in an indicator that fails to accurately reflect the Federal funding todiscovery and innovation. Also, significant fluctuations in support for “R” tendto be obscured when combined with the larger “D” category funds.

21

Life

Engineering

Physical Sciences

Environmental

Math & Computer

Social Science

Psychology

60

50

40

30

20

10

1970 1973 1976 1979 1982 1985 1988 1991 1994 19970

Percent

Over the last three decades the life sciences have come to dominate theFederal portfolio of research investments.

FIGURE 3:Shares of Federal Research by Field


CAPTURING THE CHARACTER OF ACTIVITIES SUPPORTED AT THE FIELDLEVEL

Within research, the character of research fields and activities has changedover time, resulting in definitions that no longer capture important distinctionsin federally funded research activities. Special areas of weakness includemultidisciplinary and cross-disciplinary workgroups and teams, emerging areas,differences in interpretation across agencies’ reporting units, and the evolvingcontent of traditional research fields themselves. In addition, educationalcontributions of research—particularly in academic institutions for graduateeducation—are not captured in most agencies’ databases. Current field-leveldata have the advantage of providing a time series to reveal trends in support tofields of science and engineering. New information technology is available tosupport development of richer, more easily accessible and more flexibledatabases for federally funded research activities.

RELIABILITY AND TIMELINESS

Differences in interpretation have resulted in wide discrepancies in researchfunding reported by performing and funding units—or even within the FederalGovernment across agencies and programs—even though they ostensiblydescribe the same activities. In addition, timeliness, in most cases essentialto budget allocation decisions, is not possible with Federal databases based onsurveys. Much of the data measuring the Federal research portfolio withrespect to programs funded, support for fields of science and engineering, andperforming institutions are several years old at best. Timeliness will becomeincreasingly more problematic as rapid changes in science and technologyincrease the need for current data to monitor Federal investments. Agenciesand departments could benefit from coordinated efforts across S&T fundingunits to develop a more efficient and timely data collection process while


assuring the integrity of the data they provide.

ASSESSING WORLD LEADERSHIP OF U.S. SCIENCE AND ENGINEERING

National capabilities in science and technology and the government role inenhancing these assets are growing emphases for governments around theworld. As science and technology capabilities have become more broadlydistributed, there is a need for the United States to monitor the U.S. enterpriseagainst an international backdrop to detect declines in national capabilities inscience and technology relative to other nations or to identify newopportunities for research investment that merit public support. The NationalAcademies have urged regular international benchmarking at the field level toassess the health of individual fields of research in the United States.13 Theuse of international comparisons of the productivity of research fields andinternational expert participation in assessments of research programs arecommon in other countries. The Board has noted the need for monitoring therelative health of U.S. science and technology as part of a continuing evaluationof the Federal portfolio, drawing on existing data and expert analyses, andcontinually improving data and methods for international comparisons thatinform priority setting.

UNDERSTANDING THE ROLE OF FEDERAL RESEARCH IN PRODUCINGECONOMIC AND OTHER BENEFITS

A large number of studies have attempted to elucidate, and in many casesmeasure quantitatively, the relationship between research and innovation andthe benefits of research for society. Organizations like the Council onCompetitiveness; the Science and Technology Policy Institute, RAND; OSTP;and NSF have explored issues and methods for analyzing the role of a range offactors in innovation—including federally funded research—and resultingeconomic and social benefits. On the other hand, academic programs are notdoing enough to address these questions and are inadequately funded. Thedevelopment of deeper understanding of the benefits from Federal research isan area where additional investment could improve both qualitative andquantitative data to inform budget allocation decisions, communicate thebenefits of research to the public, and contribute to the effectiveness of Federalresearch investments.

13 The National Academies, Committee on Science, Engineering and Public Policy. Science, Technology andthe Federal Government: National Goals for a New Era, 1993 and National Research Council Committeeon Criteria for Federal Support of Research and Development, Allocating Federal Funds for Science andTechnology, Appendix B,1995.

“In the long run–in goodbudget years as in bad—it isessential that policymakers...recognize the fragility of (theU.S. S&T) enterprise and thecritical Federal role insustaining it. It is up to themembers of the scientific andengineering community tocarry this message to them” –A.H. Teich, AAAS, 1999

23

SCIENCE AND TECHNOLOGY ADVICE TO CONGRESS

Congressional mechanisms that could provide review, assessment, andadvice on science and technology issues in the past included:

The Congressional Budget Office (CBO), established under theCongressional Budget Impoundment and Control Act in 1974 (PL 92-599) to provide objective, nonpartisan assistance to legislators,scores the costs of bills and prepares budget and economic forecasts;

The Congressional Research Service (CRS) provides Congress with quickresponses to a large number of requests for reports. CRS recentlymerged its Science, Technology and Medicine Division into two otherdivisions: Resources, Science and Industry (RSI) and DomesticSocial Policy (DSP);

The General Accounting Office (GAO) was established as auditor forCongress in 1921, but in the 1970s won broad authority to auditFederal programs; it was subjected to a 25 percent budget reductionin the mid 1990’s. The GAO Energy Resources and Sciences IssueArea was reorganized into the Natural Resources and EnvironmentTeam in October 2000 as part of a general reorganization;

The Office of Technology Assessment (OTA) was established bylegislation in 1972 (PL 92-484) to provide Congress with “earlyindications of the probable beneficial and adverse impacts of theapplications of technology and to develop other coordinateinformation which may assist Congress.” In total, it prepared about700 reports over 23 years.

Several of the Congressional support agencies were affected byCongressional budget cuts in the mid 1990s, with all funding eliminatedfor OTA in 1995.

Other mechanisms legislatively required to provide science and technologysupport to Federal policymakers, including Congress, are:

The National Academies, including the Academies of Science andEngineering, the Institute of Medicine, and the National ResearchCouncil;

The National Science Board; The Office of Science and Technology Policy.

Congress also employs hearings to obtain expert testimony on science andtechnology concerns.

Although the need to provide Congress with more systematic S&T review,assessment and advice has been widely supported in concept, opinionsvary on appropriate mechanisms to accomplish these ends.

BOX THREE


25

MAJOR FINDINGS AND ISSUES

The Board’s findings are based on an intensive two-year study including reviewof the literature on Federal budget coordination and priority setting for scienceand engineering research, and invited presentations from and discussions withrepresentatives of OMB, OSTP, the Federal R&D agencies, congressional staff,high level science officials from eight foreign governments, experts on data andmethodologies, and industry, the National Academies, and academicspokespersons. Discussions focused on research priority setting as it ispracticed within government organizations and suggestions on how the processmight be improved. After considering this information, the Board finds that:

Federal priority setting for research occurs at three levels: 1) establishing Federal goals for research, 2) the budget allocation processes for research within the White House

and the Congress that in the aggregate produce the Federal researchportfolio and

3) Federal agencies and departments in achieving their missions and inaccord with the President’s priorities for research.

The allocation of funds to national research goals is ultimately a politicalprocess that should be informed by the best scientific advice and data avail-able.

A strengthened process for research allocation decisions is needed. Suchallocations are based now primarily on faith in future payoffs justified bypast success, but are difficult to defend against alternative claims on thebudget that promise concrete, more easily measured results and are sup-ported by large and vocal constituencies.

The pluralistic framework for Federal research is a positive aspect of the sys-tem and increases possibilities for funding high-risk, high-payoff research.An improved process for budget coordination and priority setting should buildon strengths of the current system and focus on those weaknesses that canbe addressed by improved data and broad-based scientific input representingscientific communities and interests across all sectors.

“A primary resource thatwould provide immediatebenefits to decision makersis a broad-based, continuouscapability for expert adviceto both OMB and Congressduring the budget allocationprocess.”

CHAPTER THREE


APPROPRIATE SCIENTIFIC ADVICE

The scientific community can contribute to the Federal budget process as it nowdoes within departments, agencies and programs, by providing:

Reliable data and expert opinion on the most compelling major opportunitiesand needs for science and engineering, in the form of a well-defined set oftop research priorities for substantial additional Federal investment;

Effective processes for priority setting across fields of science and engineering,including multidisciplinary research and emerging areas;

Estimated costs and benefits of various proposals, as well as overall fundinglevels, as input to decisions;

Consensus across broad fields of research on the highest shared prioritiesfor advancing Federal goals for science and technology—through mechanismsof Federal agency advisory bodies, expert scientific staff, the NationalAcademies, and private and non-profit organizations of the research andeducation communities—to inform Federal allocation decisions.

At the Federal level, advice on priorities for major research facilities is an areafor particular attention. Facilities costs must be estimated and include long-term commitments for operation and maintenance. In addition, considerationmust be given to tradeoffs to enable funding for priority facilities.

Advice, analyses and data must be coordinated with the Executive Branch andcongressional budget processes if they are to be useful for informing researchbudget allocation decisions.

IMPROVED DATA AND ANALYSIS

Allocation decisions should be informed by available data and should employ arange of methods of analysis and data sources. Over the long term there is aneed for improvements in data, methods, and analyses that track Federal fundsand measure the costs and benefits of research. Needs include:

Improved theoretical understanding of the relationship between publiclysupported research and innovation;

There is a need for regular evaluation of Federal investments as a portfolio forsuccess in achieving Federal goals for research, to identify areas of weak-ness in national infrastructure for S&T, and to identify a well-defined set ofthe top priorities for major new research investments.

Additional resources are needed to provide both Congress and the ExecutiveBranch with data, analyses, and expert advice to inform their decisions onbudget allocations for research.

27

Improved measures of economic returns to research investments, as well asnon-economic returns in improved quality of life;

Improved understanding of the relationship between research investmentsand the S&T workforce;

Broadly acceptable definitions of “research” especially at the field level—though admittedly difficult to establish—to enable unambiguous, self-consistent tracking of Federal funds and benefits across departments,agencies and sectors;

Improved data for international comparisons, including both relative andabsolute measures; and

Improved databases and other tools for tracking research funds and measuringoutputs.

TOWARD AN ENHANCED PROCESS

The analytical and expert support available to inform research budget decisionsneed to be strengthened in both the Congress and the White House. A primaryresource that would provide immediate benefits to decision makers is a broad-based, continuous capability for expert advice to both OMB and Congress duringthe budget allocation process. A longer-term need is the regular, systematicevaluation of the effectiveness of Federal investments in achieving Federalgoals for research through OSTP, drawing broad-based input from scientificexperts and organizations in all sectors. Complementing both are improveddata and analysis on research opportunities and needs that trace Federalresearch investments through the budget process and beyond.

Strengthening the Federal mechanisms to inform research budget allocationdecisions in the White House would add an important dimension to currentmechanisms for scientific advice, which feature agency- and department-basedexternal and internal scientific input as part of their budget deliberations. Itwould require additional resources in OSTP. Additional resources might also beneeded to strengthen Congressional mechanisms to inform research budgetdecisions. Furthermore, investments in data systems and academic researchon the relationship between publicly funded research and economic and socialbenefits would enable improvements in methods for measuring and estimatingreturns on public investments. The payoff would be a more effective system forallocating Federal research funds to contribute to national goals, and improvedtools for measuring and communicating the benefits of Federal investments topolicy makers and the general public.

MAJOR FINDINGS AND ISSUES

29

CONCLUSIONS ANDRECOMMENDATIONS

Federally funded science and technology support the missions of every Federaldepartment and agency and have enormous long-term impacts on the economyand the quality of life of American citizens. The growth in the national andglobal science and technology enterprise, the opportunities for discovery andinnovation, and the changing Federal role in U.S. science and technologyrequire the Federal Government to direct greater attention to ensuring itsinvestments in research produce the greatest benefits over the long term to thepublic.

A deliberate, scientifically grounded process is essential for identifying opportu-nities and needs for Federal research. Needs include human resources, instru-mentation and facilities, alignment of the portfolio of Federal investments withnational priorities for research, effective distribution of funding among researchmodes and performing organizations, closure of gaps in research resulting fromchanges in department and agency programs, and addressing patterns of under-investment in vital areas of fundamental research.

The Board finds that mechanisms that have evolved based on the legislationthat established OSTP and on the cooperation between OSTP and OMB repre-sent valuable progress toward a more coherent and sophisticated system toinform major decisions on Federal research investments. The OMB/OSTP/PCAST must be provided with additional resources to expand activities formanaging Federal S&T as a portfolio, especially for ongoing evaluation of theeffectiveness of Federal investments in achieving Federal goals for research.Additional complementary resources to provide timely expert advice, analysesand data to inform congressional budget allocation decisions are also needed.

CHAPTER FOUR


KEYSTONE RECOMMENDATION 1

The Federal Government, including the White House, Federal depart-ments and agencies, and the Congress should cooperate in develop-ing and supporting a more productive process for allocating andcoordinating Federal research funding. The process must place apriority on investments in areas that advance important nationalgoals, identify areas ready to benefit from greater investment,address long-term needs and opportunities for Federal missions andresponsibilities, and ensure world class fundamental science andengineering capabilities across the frontiers of knowledge. Itshould incorporate input from the Federal departments and agen-cies, advisory mechanisms of the National Academies, scientificcommunity organizations representing all sectors, and a globalperspective on opportunities and needs for U.S. science and tech-nology.

RESEARCH COMMUNITY INPUT ON NEEDS ANDOPPORTUNITIES:

Steps can be taken in the short term to improve the information base forFederal research investments. A primary input to any process of priority settingfor research is expert scientific advice on current and long-term opportunitiesand needs for research. Presently there is no widely accepted and broadlyapplied way for the Federal Government to obtain systematic input from thescience and engineering communities for making priority decisions aboutsupport for research and research infrastructure.

There is insufficient opportunity and capability within the framework of existingmechanisms for Federal research priority setting to undertake timely and broad-based assessments of the needs for Federal investments. A more effectivesystem for managing the Federal research portfolio requires adequate funding,staffing and organizational continuity.

RECOMMENDATION 2

A process should be implemented that identifies priority needs andopportunities for research—encompassing all major areas of scienceand engineering—to inform Federal budget decisions. The processshould include an evaluation of the current Federal portfolio forresearch in light of national goals, and draw on: systematic,independent expert advice from the external scientificcommunities; studies of the costs and benefits of researchinvestments; and analyses of available data; and should include S&Tpriorities, advice, and analyses from Federal departments and

31

agencies. The priorities identified would inform OMB in developingits guidance to Federal departments and agencies for the President’sbudget submission, and the Congress in the budget development andappropriations processes.

EXECUTIVE BRANCH ADVISORY MECHANISM:

The Executive Branch should implement a more robust advisory mechanism,expanding on and enhancing current White House mechanisms for S&T budgetcoordination and priority setting in OSTP and OMB. Enhanced resources shouldinclude an adequate professional staff, perhaps on a rotating basis modeled onthe Council of Economic Advisors. It is particularly essential that the advisorymechanism include participants who are experienced in making choices amongexcellent opportunities or needs for research. (For example, vice provosts forresearch in universities, active researchers with breadth of vision, and manag-ers of major industrial research programs would be appropriate in this role.)

Evaluation criteria should reflect Federal goals for science and technologyfunding. The evaluation should consider the effectiveness of the broadportfolio of Federal support to science and technology for:

sustaining and enhancing U.S. world leadership across the frontiers ofknowledge;

assuring the long-term vitality of the U.S. science and technology enter-prise by investments in important areas and activities unlikely to befunded by other sectors;

aligning human resources for science and technology with needs of theS&T workforce in the Federal and other sectors;

serving Federal departmental and agency missions;

and should identify:

a well-defined set of top research priorities where enhanced Federal investments could yield high payoffs to society; and

potential tradeoffs to provide greater funding for priority activities.

CONCLUSIONS AND RECOMMENDATIONS


RECOMMENDATION 2A

An Executive Branch process for ongoing evaluation of outcomes ofthe Federal portfolio for research in light of Federal goals for S&Tshould be implemented on a five-year cycle.14 A report to thePresident and Congress should be prepared including a well-definedset of the highest long-term priorities for Federal researchinvestments. These priorities should include new nationalinitiatives, unique and paradigm shifting instrumentation andfacilities, unintended and unanticipated shifts in support amongareas of research resulting in gaps in support to important researchdomains, and emerging fields. The report should also includepotential trade-offs to provide greater funding for priority activities.The report should be updated on an annual basis as part of thebudget process, and should employ the best available data andanalyses as well as expert input. Resources available to OSTP, OMBand PCAST should be bolstered to support this function.

CONGRESSIONAL ADVISORY MECHANISM:

There is no coherent congressional mechanism for considering allocationdecisions for research within the framework of the broad Federal researchportfolio. The current system splits areas of research among numerous commit-tees and subcommittees, each considering a limited portion of the portfolio,making impossible consideration of impacts of budget allocation decisions onnational science and technology capabilities. While the need for analyticalresources for science and technology policy tailored to the congressional pro-cess has been growing, available resources have been eliminated or reduced inrecent years. And though improvements in the White House process—particu-larly expansion of activities and resources available to OSTP—would benefitCongressional allocation decisions, one or more Congressional mechanisms toprovide expert input to research allocation decisions are badly needed.

RECOMMENDATION 2B

Congress should develop appropriate mechanisms to provide it withindependent expert S&T review, evaluation, and advice. Thesemechanisms should build on existing resources for budget andscientific analysis, such as the Congressional Budget Office, theCongressional Research Service, the Government Accounting Office,and the National Academies. A framework for considering the fullFederal portfolio for science and technology might include hearingsby the Budget Committees of both houses of Congress, or other suchbroadly based congressional forums.

14 The designation of a five-year cycle for evaluation of the Federal portfolio reflects both the size of theeffort, which would require more than an annual process, and the rapid changes in science, whichdemand a frequent reevaluation of needs and opportunities for investments.

33

Advice to Congress in developing its recommendations on Federal priorities andfunding levels for research should make use of the best available data andanalyses.

DEFINITIONS, DATA AND DATA SYSTEMS:

In addition to an enhanced process for expert advice and assessment, there is along-term need to improve tools—databases and analytic methods—for effectivemanagement of the Federal research portfolio.

High quality data and data systems to monitor Federal investments in researchwould enhance the decision process. Such systems must be based ondefinitions of research activities that are consistently applied acrossdepartments and agencies and measured to capture the changing character ofresearch and research needs. Flexibility in defining categories of research fortracking purposes is especially important for monitoring emerging areas andaddressing the range of modes for research—from the individual investigator tothe major center or facility. Timely collection of data and ease of access arecritical to be useful to the allocation decision process.

Improving data and data systems is a long-term objective but one that isnecessary and increasingly urgent for managing the large, diverse Federalresearch portfolio to serve the Nation. It will require long-term commitment toimprove data systems, with input from potential users and contributors, andappropriate support.

RECOMMENDATION 3

A strategy for addressing data needs should be developed. Such astrategy supported by OMB and Congress and managed throughOSTP and OMB would assure commitment by departments, agenciesand programs to timely, accessible data that are reliable acrossreporting units and relevant to the needs for monitoring andevaluating Federal investments in research. Current data and datasystems tracking federally funded research should be evaluated forutility to the research budget allocation process and employed asappropriate.

CONCLUSIONS AND RECOMMENDATIONS


INTERNATIONAL COMPARISONS:

Both relative and absolute international statistical data and assessmentsshould be included as a major component of the information base to supportExecutive Branch and congressional research budget allocation decisions.15

International benchmarking of U.S. research performance and capabilities on aregular basis responds to the growing globalization of science and technologyand the need for the United States to maintain a world-class science andengineering infrastructure. Maintaining world-class capabilities enables theNation to take advantage of opportunities for rapid advancements in knowledgein targeted areas of research and to capitalize on breakthroughs wherever theyoccur worldwide. Although international data and methods of analysis arelimited, they should be employe

national€¦ · national science board eamon m. kelly (chair), president emeritus and professor,...

Documents