office of the science advisor peer review...office of the science advisor peer review 2006 peer...
TRANSCRIPT
![Page 1: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/1.jpg)
Office of the Science AdvisorOffice of the Science Advisor
Peer Review2006 Peer Review Policy and
3rd edition of the PR Handbook
Barbara KlieforthOffice of the Science Advisor, ORD
![Page 2: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/2.jpg)
2
Long History of Peer Review at EPA
Long History of Peer Review at EPA
1993 Peer Review Policy1998 1st Guidance Handbook2003 epa.gov/si Science Inventory2006 New Policy and 3rd edition Handbook;
http://epa.gov/osa/spc/peerreview/
![Page 3: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/3.jpg)
3
2004 OMB Peer Review BulletinKey Features
• Highly influential scientific assessments, influential scientific information
• Peer review expectations• Public comment on peer review plans• Annual reports to OMB
![Page 4: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/4.jpg)
4
Scientific Information
Factual inputs, data, models, analyses, technical information, or scientific assessments
![Page 5: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/5.jpg)
5
Influential Scientific Information (ISI)
Will have or does have a clear and substantial impact on important public policies or private sector decisions
Same definition as used in the Information Quality Guidelines• Major per the old Peer Review Handbook• Supports top Agency actions (including economically significant
actions)
![Page 6: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/6.jpg)
6
Scientific AssessmentScientific AssessmentEvaluation of a body of scientific or technical
knowledge, which typically synthesizes multiple factual inputs, data, models, assumptions, and/or applies best professional judgment to bridge uncertainties.Includes: state-of-the-science reports, technology assessments, weight-of-evidence analyses; meta-analyses; health, safety, or ecological risk assessments; toxicological characterizations of substances; integrated assessment models; hazard determinations; or exposure assessments
![Page 7: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/7.jpg)
7
Highly Influential Scientific Assessment (HISA)
Highly Influential Scientific Assessment (HISA)
• Could have a potential impact of more than $500M in any year
• Is novel, controversial, or precedent-setting, or has significant interagency interestOverlaps with definition of old “major”
![Page 8: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/8.jpg)
8
All EPA work products vs those considered ISIs or HISAs
All EPA work products vs those considered ISIs or HISAs
All work products EPA may generate or use
Influential Scientific Information
Highly Influential Scientific Assessments
![Page 9: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/9.jpg)
9
Peer Review Expectations for Other Information
Peer Review Expectations for Other Information
• Most products in Science Inventory are not ISI or HISA
• Should determine if product would benefit from peer review and, if so, what type of review to conduct
• Should also consider peer input
![Page 10: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/10.jpg)
ID work product & put in SI
Planning a Peer ReviewPlanning a Peer ReviewPlanning a Peer Review
Is it ISI or HISA?
Would it still warrant PR?
Is PR exemptor waived?
Does it consist only of science previously PR’ed?
Peer Review needed:•ID charge•ID key staff•Create PR record•Ensure funding•Determine time frame
PR Not Needed
no
yes
no
yes
no
no
![Page 11: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/11.jpg)
Conducting a Peer ReviewConducting a Peer Review
Complete PR & Obtain comments
Work ProductReady for review
Develop Charge
Select Mechanism
Set Timelines
Select Peer Reviewers
Send Materials &Conduct Review
(Add all to PR Record)
![Page 12: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/12.jpg)
Document how used &Put in PR record
Receive Comments from Peer Reviewers
Completing a Peer ReviewCompleting a Peer Review
Brief Decision-Maker, if appropriate
Are PR comments usedin Work Product?
Document why not &Put in PR record
Brief Decision-Maker on approach to handling
comments
Finalize Work Product
Update PR record
DONE!
yes
no
![Page 13: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/13.jpg)
13
Science Inventory/Peer Reviews
Science Inventory/Peer Reviews
• Science Inventory• 1995: Approximately 100 products• 2005: >5,000 activities and products
• Peer Reviews in 2004• Over 2,000 (includes journal articles)• Estimated annual cost: $10M
![Page 14: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/14.jpg)
14
Contents of Peer Review PlansContents of Peer Review Plans• Title/subject/purpose, EPA contact, if HISA or ISI• Timing of the review (including deferrals)• If panel or letter review (or through alternative procedure)• If and how public will be asked to comment on work
product• If EPA will provide public comments to peer reviewers• # of reviewers (3 or fewer, 4-10, >10)• Description of primary disciplines or expertise needed• Whether reviewers will be selected by EPA or outside
organization• If the public (including scientific/professional societies) will
be asked to nominate reviewers
![Page 15: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/15.jpg)
15
Roles/ResponsibilitiesRoles/Responsibilities• AAs/RAs• Science Policy Council• ORD/OSP• Peer Review Coordinators• Peer Review Leaders
![Page 16: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/16.jpg)
16
AA/RAAA/RA
• Is ultimately responsible for implementing policy• Determines ISI/HISA• Identifies other products needing peer review• Makes peer review decisions• Allocates resources for peer review
• Ensures that her/his office’s science activities and products are captured in the Science Inventory
![Page 17: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/17.jpg)
17
Science Policy Council (SPC)Science Policy Council (SPC)• Develops EPA science policy and provides
oversight
• Peer Review Advisory Group• Assists the SPC with implementing policy
Agency-wide• Responsible for revising Peer Review Handbook
![Page 18: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/18.jpg)
18
Peer Review CoordinatorPeer Review Coordinator• Serves as the main peer review contact for
his/her organization• Provides training on peer review process and
roles/responsibilities• Coordinates the entry and update of information in the
Science Inventory• Serves as a liaison with ORD and SPC• Assists with preparing Annual Report to OMB
![Page 19: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/19.jpg)
19
Peer Review LeaderPeer Review LeaderOrganizes/oversees peer review for a specific work
product• Selects peer review mechanism• Develops charge• Considers public comments if HISA/ISI• Documents the peer review process and results• Responds to findings• Maintains the peer review records• Works with the Coordinator on the Science
Inventory
![Page 20: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/20.jpg)
20
In SummaryISI – What is new?
In SummaryISI – What is new?
• Expanded definition of peer review products• Offer and consider public comment on peer review
plans• Consider asking public to nominate potential peer
reviewers• Preparing Peer Review Report and making it (and
related materials) available on the internet• Discussion in preamble and certification in the
administrative record
![Page 21: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/21.jpg)
21
Major Products(per old Peer Review Handbook)
Major Products(per old Peer Review Handbook)
• Establishes a significant precedent, model or methodology
• Addresses significant controversial issues• Focuses on significant emerging issues• Has significant cross-Agency/inter-agency implications• Involves significant investment of Agency resources• Considers an innovative approach for a previously
defined problem/process/methodology• Satisfies a statutory or other legal mandate for peer
review
![Page 22: Office of the Science Advisor Peer Review...Office of the Science Advisor Peer Review 2006 Peer Review Policy and 3rd edition of the PR Handbook Barbara Klieforth Office of the Science](https://reader033.vdocuments.us/reader033/viewer/2022042812/5fa95d041adceb5c8356c321/html5/thumbnails/22.jpg)
22
HISA- What is new?HISA- What is new?In addition to those listed for ISI:• When feasible, soliciting public comment on
work product and allowing public to make presentation to peer reviewers
• Providing peer reviewers with significant public comments that pertain to science issues
• Preparing and posting Agency response to peer review report