report to the biological and environmental research advisory committee by the committee of visitors...
Post on 24-Dec-2015
216 Views
Preview:
TRANSCRIPT
Report to the Biological and Environmental Research Advisory
Committee
By the Committee of Visitors for the Review of the
Climate and Environmental Sciences Division (CESD)
September 16, 2010
1
COV Charge
• On August 27, 2009, Dr. William F. Brinkman, Director for the Office of Science, charged BERAC with assembling a COV to assess the processes used to manage the Climate and Environmental Sciences Division (CESD) research programs and two user facilities.
• The solicitations, reviews, recommendations and monitoring of proposals for research submitted to CESD programs for FY2007-FY2009 were to be assessed.
2
Program Administration Changes
• Jerry Elwood, Director of CESD– Retired 2008
• Michael Kuperberg and Wanda Ferrell– Acting Directors for CESD, 2008 to August, 2010
• Gary Geernaert, new Director of CESD– August 11, 2010
3
Program Reorganizations• Climate Modeling
– Climate Response– Integrated Assessment– Climate Mitigation
• Terrestrial Systems– From Climate Forcing– From Climate Response
• Atmospheric Processes– Atmospheric Radiation
Measurement (ARM) Program– Atmospheric Science Program– ARM Climate Research Facility
• Environmental Remediation Sciences Division (ERSD)– Environmental Remediation Sciences
Program– Environmental Molecular Science
Laboratory Facility
• Climate and Earth System Modeling– Regional and Global Climate Modeling– Earth System Modeling– Integrated Assessment
• Environmental System Science– Terrestrial Ecosystem Research– Terrestrial Carbon Sequestration Research
• Atmospheric Systems Research– Atmospheric Radiation Measurement Science– Atmospheric Science
• Subsurface Biogeochemical Research Program
• Facilities– Environmental Molecular Science Laboratory – ARM Climate Research Facility
CESD 2010CESD 2007-09
4
COV MembershipCynthia Atherton* Gordon and Betty Moore Foundation
Ana P. Barros Duke University
Mark Boslough Sandia National Laboratories
Kathy Covert National Science Foundation
Robert Dickinson The University of Texas, Austin
David Hyndman Michigan State University
Bruce Kimball USDA Arid-Land Agricultural Research Center
Michael McInerney The University of Oklahoma
Steve Meador National Science Foundation
Charles Miller Jet Propulsion Laboratory
Dennis Mills Argonne National Laboratory
Robert Mitchell Joseph W. Jones Ecological Research Center
Walter Robinson NC State University
Michael Ryan USDA Forest Service
Peter Sherwood Oklahoma State University
James Schauer University of Wisconsin-Madison
Ronald Stouffer Princeton University Forrestal Campus
Paul Try Science and Technology Corporation
COV Chair – Judy Wall, University of Missouri *Unable to participate 5
COV Membership Summary• 19 scientists with representation from
– academia (9)– the private sector (1)– the National Laboratories (ANL and SNL) (2)– private foundations (2)– the Federal Government (5)
• 2 from the National Science Foundation• 2 from the US Department of Agriculture• 1 scientist from the National Aeronautics and Space Administration
• 5 of the COV members have served on prior COV reviews or reviews for the Office of Project Assessment
• 2 of the COV members are also members of BERAC
6
The COV Meeting
• The COV met July 20-22, 2010 at the DOE headquarters in Germantown, Maryland
• 6 subcommittees reviewed the four research areas and two facilities
• The Program Managers provided all material requested, were available throughout the COV visit, presented visual summaries of the programs and patiently answered all questions.
7
Program Managers
KnowledgeablePassionate
Tireless
The COV commends the work of these dedicated individuals in guiding and overseeing the broad array of critical research programs in the CESD.
8
General Comments and Recommendations
• The scientific portfolio of the CESD was appropriate to the goals of DOE.
• The research addressed critical concerns and has made substantive contributions to fundamental knowledge in these areas.
• Because the unique research capabilities of the National Laboratories differ from those available in universities or smaller research clusters, a different program and funding arrangement has been implemented to allow their potential to be realized. These are now Science Focus Areas (SFA).
The CESD is to be commended for continued support of critical research in areas of national concern and creativity in their administration.
9
General Comments and Recommendations: Program Managers
The prior COV stated, “We find that insufficient resources, both in terms of personnel and budgetary support, collectively place at risk the ability of the Program Managers to effectively manage and oversee the programs for which they have responsibility.”
This is a continuing and critical concern of the current COV.
10
• The National Laboratories are transitioning to SFAs.
• This is creating a duel evaluation and reporting system for the PMs.
• No additional resources for the increased administrative tasks have been provided.
The COV recommends that more staff be made available for workshop and review planning as well as reviewer database maintenance.
General Comments and Recommendations: Program Managers
11
• The peer review process is working well.• Preproposals, that save time for PIs and reviewers, are
almost universally used by the PMs in this division.• Reviews are thorough and timely.• Reviewers are appropriate and without apparent biases.• PMs often communicate personally with PIs to resolve
reviewer questions. • However, feedback on unsuccessful proposals from PMs is
often not substantive.The COV recommends more informative statements be
included in declination letters to guide appropriate responses by PIs. This effort is likely a casualty of time and energy constraints.
General Comments and Recommendations: Program Managers
12
• The COV recommends improvement in the electronic grant record keeping at DOE.
• More effort is encouraged for the publication of DOE contributions to make the public aware of the high quality of work supported.
• As research programs evolve, program name changes can become confusing. Care should be taken to delineate the relationships between new and older programs.
General Comments and Recommendations:
13
• The SFA funding model for programs of research at the National Laboratories raises concerns regarding– Nimble responses to research priority changes– Efficient use of resources– Decisions regarding distribution of resources
between SFAs and research efforts of other agency labs and universities.
The COV recommends that a plan for recompeting SFAs be developed and implemented as soon as conveniently possible.
General Comments and Recommendations:
14
Atmospheric System Research
“Quantify the interactions among aerosols, clouds, precipitation, radiation, dynamics, and thermodynamics to improve fundamental process-level understanding, with the ultimate goal to reduce the uncertainty in global and regional climate simulations and projections.”
15
Atmospheric System Research• Three solicitations were evaluated.– All proposals were evaluated by mail review– Reviewers were appropriate– Reviews were of high quality– In one solicitation, all new applications appeared to be
declined. This would suggest that the call could be improved to avoid unnecessary effort.
The COV recommends improved solicitations and better use of pre-
proposal process, panel reviews of proposals and quantitative metrics of output publications to
improve monitoring and as input to indicate future research directions.
16
Atmospheric System Research
• One SFA application was evaluated– Review was rigorous– Research was cutting-edge
• It is not clear what the future management will be and whether competitive assessments are planned.
As transition to SFA funding is initiated, the COV supports the design of an ongoing review schedule that assures long-term excellence.
17
Climate Modeling Programs• The components of this program include:– Climate Modeling (Earth System Modeling)– Integrated Assessment– Regional and Global Climate Modeling
• Five solicitations were evaluated• Only two PMs manage this effort
18
Climate Modeling ProgramsThe COV was impressed with the review process,
the breadth and quality of the programs and the international impact of the findings.
Gender balance was good in the awards made with the exception that all National Lab PIs were men, reflecting the demographics of senior scientists there.
The COV lauds the PMs’ organization of a PI meeting that is proposed to become an annual event.
19
Climate Modeling Programs• Includes critical components of the overall US
and international global climate change programs– Working with DOE Office of Advanced Scientific
Computing Research to provide most advanced computational capabilities for the next Intergovernmental Panel on Climate Change (IPCC) assessment.
– The Program for Climate Model Diagnosis and Intercomparison (PCMDI) and the Earth System Grid now provides data for the international community.
20
Climate Modeling Programs• Vital for effective climate modeling is reliable access
to cutting-edge computing software and capacity at the decadal level. Currently high-performance computing is appropriately decoupled from Climate Modeling projects, but allocation of these resources requires a systematic mechanism currently lacking.
• There are too few PMs for this large program
• A mentoring program for new PMs is recommended.
21
Subsurface Biogeochemical Research • Formerly the Environmental Research Science
Program and a separate division.
• Two university and two SFA solicitations were reviewed.
22
Subsurface Biogeochemical Research • The COV found the review of solicitations to
be very well handled resulting in selection of high quality research projects.
• Declination letters were not particularly informative.
While the COV notes that all PIs receive all reviewer comments, it is recommended that sufficient information be included in decision letters for PI decision making about resubmission.
23
Subsurface Biogeochemical Research • SFA solicitations were models for implementation– Science focus planning – Expert panel reviews– Annual reports required– Three year reviews with a rolling schedule developed– The plan for continuation beyond six years was not clearThe COV recommends that plans for year six and beyond
be articulated and communicated.Additional reviews might take advantage of virtual
technology.
24
Subsurface Biogeochemical Research • Included in the SBR program are the Science Discovery
through Advanced Computing (SciDAC) and the Integrated Field Research Challenge (IFRC).
• Management of SciDAC has been good and partnership with ASCR effective.
• The IFRC sites were well reviewed when funded.• All are highly productive and serve to integrate
scientist efforts.The COV recommends development of a comprehensive
data management plan for the IFRCs.Recompeting should be considered.
25
Terrestrial Ecosystem Science/Terrestrial Carbon Sequestration
• Terrestrial Carbon Processes and Ecosystem Function and Response programs were consolidated to form TES.
• Climate Mitigation transitioned to Terrestrial Carbon Sequestration.
• One university and one SFA solicitation were evaluated.
26
Terrestrial Ecosystem Science/Terrestrial Carbon Sequestration
• Accomplishments– Suite of free-air CO2 experiments– The CO2 – ozone interaction experiment– The precipitation manipulation experiments– Temperature manipulation experiments– Established long-term flux measuring sites– Ameriflux network– Important understanding of belowground and
decomposition research
27
Terrestrial Ecosystem Science/Terrestrial Carbon Sequestration
• The reorganization was considered worthwhile and appropriate.
• Noteworthy was the PM’s enthusiasm about the potential of the program.
The COV applauds efforts to discontinue renewals with little critical evaluation and supports adoption of competitive processes that are transparent, rigorous and well documented.
The COV suggests that solicitations should target model needs or deficiencies as a selection criterion for proposal funding.
A workshop is recommended to determine how ecosystem models can be better interfaced into climate models. 28
Terrestrial Ecosystem Science/Terrestrial Carbon Sequestration
• This program manages Ameriflux, the network of micrometeorological tower sites providing continuous observations of ecosystem level parameters across North, Central and South America.
• Ameriflux is no longer monitored with standard research proposals.
• The COV was concerned that Ameriflux support would compete with that available for research efforts.
The COV recommends careful consideration of the long-term commitment to Ameriflux and its goals and suggests site reviews be considered.
29
Atmospheric Radiation Measurement (ARM) Climate Research Facility (ACRF)
• The ACRF is a multiplatform national scientific user facility for obtaining continuous field measurements of climate data.
• Three site types:– Fixed sites in diverse climate regions– Mobile facilities (AMF)– Aerial facilities (AAF)
• Two ACRF infrastructure funding calls, open only to National Labs, were reviewed. Single awards made.– ARM Aerial Vehicle Program– AMF for marine science
30
ARM Climate Research Facility (ACRF)
• Program has been quite successful• During the 2007-09 window, 235 papers were
published with data from ACRF.• Funding and management has been separated
from the research program of ARM.• For ACRF solicitations, reviews were thorough
and reviewers knowledgeable.• Reviews were all mail in.
31
ARM Climate Research Facility (ACRF)
The COV would suggest a face-to-face panel or telecon for the technical merit review.
With the removal of ARM oversight, the COV has some concern that ACRF will suffer from the lack of scientific feedback and interactions with user PIs.
The COV praises the ACRF for its proactive management in developing the “best estimate” cloud/aerosol data sets and encourages this approach in other areas.
32
Environmental and Molecular Sciences Laboratory (EMSL)
• The EMSL is a unique user facility where a variety of tools are available to scientists for multi-level investigation of processes that underpin the goals of the DOE and the Nation.
• This facility has been removed from the ERSP (now SBR) oversight.
• Proposals are reviewed in house at EMSL and reviews were not available.
33
Environmental and Molecular Sciences Laboratory (EMSL)
• Proposals are of four types:– Science Theme (60%) funded up to 3 yrs– General, submitted anytime– Partner, cost shared– Rapid, short-term up to a month
• Reviews have both internal and external components and enumerated criteria.
• In FY09, 80 of 122 proposals were approved.• Guidelines for preparation of proposals are not rigidly enforced.
The COV recommends that length guidelines be adhered to for fairness to PIs.
Additional effort to engage the private sector as users was recommended.
34
Environmental and Molecular Sciences Laboratory (EMSL)
• DOE/BER review their user facilities on a three-year cycle.
• EMSL was reviewed in 2008 and five issues were identified that required formal action.
• A plan of action was established and corrections have been made.
The COV praises the quick and thorough response of both EMSL and DOE to the recommendations of the 2008 science and operations review.
35
top related