exxonmobil biomedical sciences, inc. (ljh) sra / fda peer review workshop - 9/30/03 peer review:...
TRANSCRIPT
ExxonMobil Biomedical Sciences, Inc.(LJH) SRA / FDA Peer Review Workshop - 9/30/03
Peer Review:
Challenges Raised by OMB’s Draft Guidelines
Leslie Hushka, Ph.D., DABTSeptember 30, 2003
(LJH) SRA / FDA Peer Review Workshop - 9/30/03ExxonMobil Biomedical Sciences, Inc.
22
ExxonMobil Biomedical Sciences, Inc.
(LJH) SRA / FDA Peer Review Workshop - 9/30/03ExxonMobil Biomedical Sciences, Inc.
33
What Guideline Elements Are Addressed
Bulletin Goals
Information Covered / Not Covered
Selecting Peer Reviewers
Peer Review Process
Information Access and Public Comments
Updating Agency Guidelines
Summary
(LJH) SRA / FDA Peer Review Workshop - 9/30/03ExxonMobil Biomedical Sciences, Inc.
44
What is Not Covered in This Section
Dr. Richard Belzer, President Regulatory Checkbook
Scholarly peer review Government peer review (including ownership, objectives, selection, procedures, interests, accountability, etc)
No evidence that peer review is an appropriate remedy for the "problem" government peer review is supposed to solve
Too much attention is devoted to conflicts of interest and not enough to coincidence of interest
Dr. Steven Lewis, Distinguished Scientific Associate, EMBSI
Peer Review Stakeholder Dialogue
Conflict of Interest Bias
Peer Review Sound science
SRA Forum on Peer Review - May 29 2002
(LJH) SRA / FDA Peer Review Workshop - 9/30/03ExxonMobil Biomedical Sciences, Inc.
55
Bulletin Goals... Ensure that agencies conduct peer reviews of the most important scientific & technical information
... Peer reviews are reliable, independent and transparent
Questions:Are these the proper goals? Are these aligned with info quality?
What is reliable?
Independent? From Agency only? Transparent how & when?
What criteria will be used to judge success in meeting goals?
(LJH) SRA / FDA Peer Review Workshop - 9/30/03ExxonMobil Biomedical Sciences, Inc.
66
Information that is Covered / Not Covered by Guidelines
... Regulatory information means any scientific or technical study that is relevant to regulatory policy . . . used by regulatory bodies
... Peer review undertaken by a scientific journal may be generally be presumed to be adequate
Questions:Does journal peer review meet the amended PR - IQG standards?
What if the Agency re-analyzes, summarizes, interpretes original findings?
Does establishing a rebuttable presumption for published articles free agencies from requirement to substantiate the quality of information?
(LJH) SRA / FDA Peer Review Workshop - 9/30/03ExxonMobil Biomedical Sciences, Inc.
77
Selecting Peer Reviewers
... Selected on the basis of necessary scientific and technical expertise ... Broad a range of expertise as is necessary
... Do not possess real or perceived conflicts on interest
Questions:Will this create additional burden or discourage qualified scientists?
Is a broad range of expertise really necessary? Does it dilute skills & decrease rigor of review?
Conflicts of interest and bias - Does the 'appearance' of either of these affect selection of reviewers?
How will the agency measure 'perceived' conflicts of interest?
(LJH) SRA / FDA Peer Review Workshop - 9/30/03ExxonMobil Biomedical Sciences, Inc.
88
Peer Review Process
...Select peer review mechanism based on the novelty and complexity of the science to be reviewed, the benefits and cost implications, and any
’controversy’ regarding the science
Questions:Are 'controversies' on the science a valid criteria for selecting a peer review
method? Is ‘uncertainty’ a more appropriate term?
Agency must provide reviewers with information on those potential sources of controversy . . .What and how?
If Agency uses other firm(s) to conduct review, does it establish some dependence upon the Agency?
(LJH) SRA / FDA Peer Review Workshop - 9/30/03ExxonMobil Biomedical Sciences, Inc.
99
Information Access and Public Comments
...Provide an opportunity for other interested agencies and persons to submit comments ... provided to peer reviewers
....Disclose names, qualifications of peer reviewers...Include a certification explaining how agency has complied
Questions:How will the Agency address public & peer review comments on information products that have
not been 'disseminated'? How will agencies set up an administrative records for information products that are years away
from use in rulemakings?
What criteria will OMB use to review ‘certifications’ for PR/IQG? Should these criteria be the same to evaluate if a review is successful? Can criteria be outcome-based and not output-focused?
(LJH) SRA / FDA Peer Review Workshop - 9/30/03ExxonMobil Biomedical Sciences, Inc.
1010
Updated Agency Guidelines
...Supplement or amend IQGs to incorporate requirements...Guidelines for... entanglements that preclude an individuals
... Confidentiality in peer review
Questions:Performance standards or detailed processes? Is this appropriate? How will
performance be measured?
Should separate agency guidelines / standards be developed by each agency?
Any criteria for excluding peer reviewers?
What, if anything, is confidential in the peer review process?
(LJH) SRA / FDA Peer Review Workshop - 9/30/03ExxonMobil Biomedical Sciences, Inc.
1111
Data “Quality” Contains Multiple Elements
Objectivity Present info in accurate, clear, complete and
unbiased manner Identify sources, supporting data and models Ensure transparency, reproducibility, and use
“best available science”
Utility Agency must consider how the
public uses the information and ensure appropriate transparency for those uses
Integrity Ensure that information is not
compromised through corruption or falsification
(LJH) SRA / FDA Peer Review Workshop - 9/30/03ExxonMobil Biomedical Sciences, Inc.
1212
Summary
Many questions for agencies to address in developing or modifying peer review programs
Overriding questions:Will peer review improve the quality of information disseminated?
If so, will peer review also improve the underlying quality of science used for regulatory purposes?
If not, what will?
Are there other models that will better fit these needs?