Report to the Biological and Environmental Research Advisory

Committee

By the Committee of Visitors for the Review of the

Climate and Environmental Sciences Division (CESD)

September 16, 2010

1

COV Charge

• On August 27, 2009, Dr. William F. Brinkman, Director for the Office of Science, charged BERAC with assembling a COV to assess the processes used to manage the Climate and Environmental Sciences Division (CESD) research programs and two user facilities.

• The solicitations, reviews, recommendations and monitoring of proposals for research submitted to CESD programs for FY2007-FY2009 were to be assessed.

2

Program Administration Changes

• Jerry Elwood, Director of CESD– Retired 2008

• Michael Kuperberg and Wanda Ferrell– Acting Directors for CESD, 2008 to August, 2010

• Gary Geernaert, new Director of CESD– August 11, 2010

3

Program Reorganizations• Climate Modeling

– Climate Response– Integrated Assessment– Climate Mitigation

• Terrestrial Systems– From Climate Forcing– From Climate Response

• Atmospheric Processes– Atmospheric Radiation

Measurement (ARM) Program– Atmospheric Science Program– ARM Climate Research Facility

• Environmental Remediation Sciences Division (ERSD)– Environmental Remediation Sciences

Program– Environmental Molecular Science

Laboratory Facility

• Climate and Earth System Modeling– Regional and Global Climate Modeling– Earth System Modeling– Integrated Assessment

• Environmental System Science– Terrestrial Ecosystem Research– Terrestrial Carbon Sequestration Research

• Atmospheric Systems Research– Atmospheric Radiation Measurement Science– Atmospheric Science

• Subsurface Biogeochemical Research Program

• Facilities– Environmental Molecular Science Laboratory – ARM Climate Research Facility

CESD 2010CESD 2007-09

4

COV MembershipCynthia Atherton* Gordon and Betty Moore Foundation

Ana P. Barros Duke University

Mark Boslough Sandia National Laboratories

Kathy Covert National Science Foundation

Robert Dickinson The University of Texas, Austin

David Hyndman Michigan State University

Bruce Kimball USDA Arid-Land Agricultural Research Center

Michael McInerney The University of Oklahoma

Steve Meador National Science Foundation

Charles Miller Jet Propulsion Laboratory

Dennis Mills Argonne National Laboratory

Robert Mitchell Joseph W. Jones Ecological Research Center

Walter Robinson NC State University

Michael Ryan USDA Forest Service

Peter Sherwood Oklahoma State University

James Schauer University of Wisconsin-Madison

Ronald Stouffer Princeton University Forrestal Campus

Paul Try Science and Technology Corporation

COV Chair – Judy Wall, University of Missouri *Unable to participate 5

COV Membership Summary• 19 scientists with representation from

– academia (9)– the private sector (1)– the National Laboratories (ANL and SNL) (2)– private foundations (2)– the Federal Government (5)

• 2 from the National Science Foundation• 2 from the US Department of Agriculture• 1 scientist from the National Aeronautics and Space Administration

• 5 of the COV members have served on prior COV reviews or reviews for the Office of Project Assessment

• 2 of the COV members are also members of BERAC

6

The COV Meeting

• The COV met July 20-22, 2010 at the DOE headquarters in Germantown, Maryland

• 6 subcommittees reviewed the four research areas and two facilities

• The Program Managers provided all material requested, were available throughout the COV visit, presented visual summaries of the programs and patiently answered all questions.

7

Program Managers

KnowledgeablePassionate

Tireless

The COV commends the work of these dedicated individuals in guiding and overseeing the broad array of critical research programs in the CESD.

8

General Comments and Recommendations

• The scientific portfolio of the CESD was appropriate to the goals of DOE.

• The research addressed critical concerns and has made substantive contributions to fundamental knowledge in these areas.

• Because the unique research capabilities of the National Laboratories differ from those available in universities or smaller research clusters, a different program and funding arrangement has been implemented to allow their potential to be realized. These are now Science Focus Areas (SFA).

The CESD is to be commended for continued support of critical research in areas of national concern and creativity in their administration.

9

General Comments and Recommendations: Program Managers

The prior COV stated, “We find that insufficient resources, both in terms of personnel and budgetary support, collectively place at risk the ability of the Program Managers to effectively manage and oversee the programs for which they have responsibility.”

This is a continuing and critical concern of the current COV.

10

• The National Laboratories are transitioning to SFAs.

• This is creating a duel evaluation and reporting system for the PMs.

• No additional resources for the increased administrative tasks have been provided.

The COV recommends that more staff be made available for workshop and review planning as well as reviewer database maintenance.

General Comments and Recommendations: Program Managers

11

• The peer review process is working well.• Preproposals, that save time for PIs and reviewers, are

almost universally used by the PMs in this division.• Reviews are thorough and timely.• Reviewers are appropriate and without apparent biases.• PMs often communicate personally with PIs to resolve

reviewer questions. • However, feedback on unsuccessful proposals from PMs is

often not substantive.The COV recommends more informative statements be

included in declination letters to guide appropriate responses by PIs. This effort is likely a casualty of time and energy constraints.

General Comments and Recommendations: Program Managers

12

• The COV recommends improvement in the electronic grant record keeping at DOE.

• More effort is encouraged for the publication of DOE contributions to make the public aware of the high quality of work supported.

• As research programs evolve, program name changes can become confusing. Care should be taken to delineate the relationships between new and older programs.

General Comments and Recommendations:

13

• The SFA funding model for programs of research at the National Laboratories raises concerns regarding– Nimble responses to research priority changes– Efficient use of resources– Decisions regarding distribution of resources

between SFAs and research efforts of other agency labs and universities.

The COV recommends that a plan for recompeting SFAs be developed and implemented as soon as conveniently possible.

General Comments and Recommendations:

14

Atmospheric System Research

“Quantify the interactions among aerosols, clouds, precipitation, radiation, dynamics, and thermodynamics to improve fundamental process-level understanding, with the ultimate goal to reduce the uncertainty in global and regional climate simulations and projections.”

15

Atmospheric System Research• Three solicitations were evaluated.– All proposals were evaluated by mail review– Reviewers were appropriate– Reviews were of high quality– In one solicitation, all new applications appeared to be

declined. This would suggest that the call could be improved to avoid unnecessary effort.

The COV recommends improved solicitations and better use of pre-

proposal process, panel reviews of proposals and quantitative metrics of output publications to

improve monitoring and as input to indicate future research directions.

16

Atmospheric System Research

• One SFA application was evaluated– Review was rigorous– Research was cutting-edge

• It is not clear what the future management will be and whether competitive assessments are planned.

As transition to SFA funding is initiated, the COV supports the design of an ongoing review schedule that assures long-term excellence.

17

Climate Modeling Programs• The components of this program include:– Climate Modeling (Earth System Modeling)– Integrated Assessment– Regional and Global Climate Modeling

• Five solicitations were evaluated• Only two PMs manage this effort

18

Climate Modeling ProgramsThe COV was impressed with the review process,

the breadth and quality of the programs and the international impact of the findings.

Gender balance was good in the awards made with the exception that all National Lab PIs were men, reflecting the demographics of senior scientists there.

The COV lauds the PMs’ organization of a PI meeting that is proposed to become an annual event.

19

Climate Modeling Programs• Includes critical components of the overall US

and international global climate change programs– Working with DOE Office of Advanced Scientific

Computing Research to provide most advanced computational capabilities for the next Intergovernmental Panel on Climate Change (IPCC) assessment.

– The Program for Climate Model Diagnosis and Intercomparison (PCMDI) and the Earth System Grid now provides data for the international community.

20

Climate Modeling Programs• Vital for effective climate modeling is reliable access

to cutting-edge computing software and capacity at the decadal level. Currently high-performance computing is appropriately decoupled from Climate Modeling projects, but allocation of these resources requires a systematic mechanism currently lacking.

• There are too few PMs for this large program

• A mentoring program for new PMs is recommended.

21

Subsurface Biogeochemical Research • Formerly the Environmental Research Science

Program and a separate division.

• Two university and two SFA solicitations were reviewed.

22

Subsurface Biogeochemical Research • The COV found the review of solicitations to

be very well handled resulting in selection of high quality research projects.

• Declination letters were not particularly informative.

While the COV notes that all PIs receive all reviewer comments, it is recommended that sufficient information be included in decision letters for PI decision making about resubmission.

23

Subsurface Biogeochemical Research • SFA solicitations were models for implementation– Science focus planning – Expert panel reviews– Annual reports required– Three year reviews with a rolling schedule developed– The plan for continuation beyond six years was not clearThe COV recommends that plans for year six and beyond

be articulated and communicated.Additional reviews might take advantage of virtual

technology.

24

Subsurface Biogeochemical Research • Included in the SBR program are the Science Discovery

through Advanced Computing (SciDAC) and the Integrated Field Research Challenge (IFRC).

• Management of SciDAC has been good and partnership with ASCR effective.

• The IFRC sites were well reviewed when funded.• All are highly productive and serve to integrate

scientist efforts.The COV recommends development of a comprehensive

data management plan for the IFRCs.Recompeting should be considered.

25

Terrestrial Ecosystem Science/Terrestrial Carbon Sequestration

• Terrestrial Carbon Processes and Ecosystem Function and Response programs were consolidated to form TES.

• Climate Mitigation transitioned to Terrestrial Carbon Sequestration.

• One university and one SFA solicitation were evaluated.

26

Terrestrial Ecosystem Science/Terrestrial Carbon Sequestration

• Accomplishments– Suite of free-air CO2 experiments– The CO2 – ozone interaction experiment– The precipitation manipulation experiments– Temperature manipulation experiments– Established long-term flux measuring sites– Ameriflux network– Important understanding of belowground and

decomposition research

27

Terrestrial Ecosystem Science/Terrestrial Carbon Sequestration

• The reorganization was considered worthwhile and appropriate.

• Noteworthy was the PM’s enthusiasm about the potential of the program.

The COV applauds efforts to discontinue renewals with little critical evaluation and supports adoption of competitive processes that are transparent, rigorous and well documented.

The COV suggests that solicitations should target model needs or deficiencies as a selection criterion for proposal funding.

A workshop is recommended to determine how ecosystem models can be better interfaced into climate models. 28

Terrestrial Ecosystem Science/Terrestrial Carbon Sequestration

• This program manages Ameriflux, the network of micrometeorological tower sites providing continuous observations of ecosystem level parameters across North, Central and South America.

• Ameriflux is no longer monitored with standard research proposals.

• The COV was concerned that Ameriflux support would compete with that available for research efforts.

The COV recommends careful consideration of the long-term commitment to Ameriflux and its goals and suggests site reviews be considered.

29

Atmospheric Radiation Measurement (ARM) Climate Research Facility (ACRF)

• The ACRF is a multiplatform national scientific user facility for obtaining continuous field measurements of climate data.

• Three site types:– Fixed sites in diverse climate regions– Mobile facilities (AMF)– Aerial facilities (AAF)

• Two ACRF infrastructure funding calls, open only to National Labs, were reviewed. Single awards made.– ARM Aerial Vehicle Program– AMF for marine science

30

ARM Climate Research Facility (ACRF)

• Program has been quite successful• During the 2007-09 window, 235 papers were

published with data from ACRF.• Funding and management has been separated

from the research program of ARM.• For ACRF solicitations, reviews were thorough

and reviewers knowledgeable.• Reviews were all mail in.

31

ARM Climate Research Facility (ACRF)

The COV would suggest a face-to-face panel or telecon for the technical merit review.

With the removal of ARM oversight, the COV has some concern that ACRF will suffer from the lack of scientific feedback and interactions with user PIs.

The COV praises the ACRF for its proactive management in developing the “best estimate” cloud/aerosol data sets and encourages this approach in other areas.

32

Environmental and Molecular Sciences Laboratory (EMSL)

• The EMSL is a unique user facility where a variety of tools are available to scientists for multi-level investigation of processes that underpin the goals of the DOE and the Nation.

• This facility has been removed from the ERSP (now SBR) oversight.

• Proposals are reviewed in house at EMSL and reviews were not available.

33

Environmental and Molecular Sciences Laboratory (EMSL)

• Proposals are of four types:– Science Theme (60%) funded up to 3 yrs– General, submitted anytime– Partner, cost shared– Rapid, short-term up to a month

• Reviews have both internal and external components and enumerated criteria.

• In FY09, 80 of 122 proposals were approved.• Guidelines for preparation of proposals are not rigidly enforced.

The COV recommends that length guidelines be adhered to for fairness to PIs.

Additional effort to engage the private sector as users was recommended.

34

Environmental and Molecular Sciences Laboratory (EMSL)

• DOE/BER review their user facilities on a three-year cycle.

• EMSL was reviewed in 2008 and five issues were identified that required formal action.

• A plan of action was established and corrections have been made.

The COV praises the quick and thorough response of both EMSL and DOE to the recommendations of the 2008 science and operations review.

35

The COV wishes to THANK!• Anna Palmisano, Director of BER• Wanda Ferrell, Acting Director of CESD• Todd Anderson Liaisons for the COV• Mike Kuperberg• All the CESD Program Managers!

• Special thanks to:• Eileen Knox, BER• Karen Carlson-Brown, BER• Leslie Runion, BER• Tracey Vieser, ORISE

36