Download - NIH Review Procedures
![Page 1: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/1.jpg)
NIH Review Procedures
Betsy Myers
Hospital for Special Surgery
![Page 2: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/2.jpg)
General Process for Proposal Review
• Applicant has idea
• Forms are submitted
• Proposal is reviewed for scientific merit
• Summary statement is prepared
• Funding Institute or Center gives information to applicant
www.csr.nih.govwww.csr.nih.gov
![Page 3: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/3.jpg)
Receipt and Review Process at NIH
• >10,000 applications arrive at given deadline!
• Receipt/review process organized in 3 cycles per year
![Page 4: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/4.jpg)
• Assignment based on Abstract, Specific Aims, more if needed
• Each application assigned to funding Institute(s)/Center(s)
• Application also assigned to Integrated Review Group (IRG) within Center for Scientific Review (CSR) or to Institute/Center (IC) review group
Assignment Process at NIH
![Page 5: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/5.jpg)
• CSR:
–R01s, R03s, R21s, Small business, Fellowships
–Reviews for >1 Institute
• IC Review:
–Program projects, Training grants, Career development awards, Responses to Requests for Applications
–Specific to Institute
Assignment Process at NIH
![Page 6: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/6.jpg)
• Application then assigned to Study Section
• NIH officials will consider requests for these assignments
–Cover letter
Assignment Process at NIH
![Page 7: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/7.jpg)
Process at NIH
Scientific Review Administrator (SRA) of Study Section decides on reviewers from within members of Study Section or from ad hoc members
![Page 8: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/8.jpg)
Appointment of Reviewers to Study Section
• SRA recruits members of Study Section
• Qualifications
–Expert with training and experience in relevant scientific field
• Level of formal education
• Quantity and quality of relevant research
–PI on research project comparable to those being reviewed
![Page 9: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/9.jpg)
Appointment of Reviewers to Study Section
SRA also needs to address
–Diversity in gender, race, ethnicity and geographic distribution
–Fairness and evenhandedness in review
–Willingness to do the work required
–Ability to write and present clearly
![Page 10: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/10.jpg)
Types of appointments to study section
–Regular: Typically several years
–Temporary: One time on standing study section, may lead to regular appointment
–Special emphasis panel: One time only
Appointment of Reviewers to Study Section
![Page 11: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/11.jpg)
Roster of Study Section Available Online
Meeting Roster - ZRG1 MOSS-A 91 (4/6/2005 - 4/6/2005)
CENTER FOR SCIENTIFIC REVIEWSPECIAL EMPHASIS PANEL
ZRG1 MOSSA4/6/2005-4/6/2005
MEETING ROSTER
CHAIRPERSON--------------------------------------MYERS, ELIZABETH R. , PHDASSOCIATE PROFESSORWEILL MEDICAL COLLEGE OF CORNELL UNIVERSITYASSOCIATE SCIENTIST, HOSPITAL FOR SPECIAL SURGERYNEW YORK, NY 10021
MEMBERS----------------ADAMS, JOHN S. , MDPROFESSORDEPARTMENT OF MEDICINECEDARS-SINAI MEDICAL CENTERLOS ANGELES, CA 90048
AHLGREN, SARA C. , PHD…
http://www.csr.nih.gov/Committees/rosterindex.asphttp://www.csr.nih.gov/Committees/rosterindex.asp
![Page 12: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/12.jpg)
• SRA matches grant applications to specific reviewers
• Tries to ensure–Appropriate expertise
–Diverse scientific viewpoints
• Tries to avoid–Overload of particular reviewer
–Potential conflict of interest
Assignment of Applications to Reviewers
![Page 13: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/13.jpg)
2 reviewers and 1 discussant (typically) are assigned to each proposal
–Primary reviewer
–Secondary reviewer
–Reader (does not need to prepare written review prior to meeting of study section)
Could be more – Tertiary, more Readers
Assignment of Applications to Reviewers
![Page 14: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/14.jpg)
NIH Review Criteria
NIH review criteria for unsolicited research project grant applications (R01, R03, R21)
• Significance
–Important problem
–Advancement of scientific knowledge or clinical practice
–Influence on methods that drive the field
![Page 15: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/15.jpg)
NIH Review Criteria
• Approach
–Adequate development and integration of design, methods, analyses
–Acknowledgment of potential problems, alternatives
• Innovation
–Challenge to existing paradigms
–Novel concepts, approaches, methods
![Page 16: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/16.jpg)
NIH Criteria
• Investigator
–Appropriate training, experience
–Complementary and integrated team
• Environment
–Conducive to probability of success
–Unique features of scientific environment
–Institutional support
![Page 17: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/17.jpg)
NIH Criteria
• Other criteria
–Gender/minority/children inclusion
–Budget
–Protection of humans, animals, and environment
• Overall rating
–Numerical score that reflects overall impact
![Page 18: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/18.jpg)
REVIEW CRITERIA: “K” Awards
• Candidate
• Career Development Plan
• Research Plan
• Mentor/Co-Mentor(s)
• Environment & Institutional Commitment
![Page 19: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/19.jpg)
NIH Numerical Rating
Priority score: Single, global score for proposal
5.0 4.0 3.0 2.0 1.0
HighestPriority
Strong in all categories
Averageapplication
WORST BEST
LowestPriority
![Page 20: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/20.jpg)
Guide to Calibrating Score
Score Descriptors
1.0 – 1.5 Outstanding, Close to flawless
1.6 – 2.0 Highly significant, Few weaknesses
2.1 – 2.5 Weaknesses need to be addressed
2.6 – 3.0 Weaknesses balance strengths
>3.0 Weaknesses outweigh strengths
![Page 21: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/21.jpg)
Submitting Critique Before Meeting
Electronic submission of reviews
• Several days before meeting, reviewers upload score and written critique
• Once uploaded, can then read other reviewers’ scores and reviews
• Once uploaded, reviewer cannot make changes to scores or critiques until after meeting
![Page 22: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/22.jpg)
Study Section Meeting
Streamlined Applications
• Definition:
–Not in upper half
–Priority score higher than 3
• Does not apply to career awards, fellowships (R13, R18, F06, F32/33)
5.0 4.0 3.0 2.0 1.0
![Page 23: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/23.jpg)
Study Section MeetingStreamlining
Streamlining Procedure
• Reviewers asked ahead of time to recommend applications not in upper half (“unscored” or “streamlined”)
• SRA compiles list
• List discussed at beginning of meeting
• Any member may ask for proposal to be discussed
![Page 24: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/24.jpg)
Benefits and rationale
• Gives time for in-depth discussion of better applications
• Saves costs if meeting is shortened
• Reduces work of scientific review administrators
Less than 25% of applications will be funded
Study Section MeetingStreamlining
![Page 25: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/25.jpg)
• If application is streamlined, applicant receives unaltered written critiques
• Fate of unscored applications?
Study Section MeetingStreamlining
![Page 26: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/26.jpg)
Study Section MeetingReview Procedures
Review procedure for proposals to be scored
–Chair introduces application
–Each reviewer gives preliminary numerical score or range
–Primary reviewer covers description and comments
–Other assigned reviewers add comments
![Page 27: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/27.jpg)
Review procedure, continued
–Discussion ensues
–Consensus is not necessary
–Chair calls for priority rating
–Every members scores
Study Section MeetingReview Procedures
![Page 28: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/28.jpg)
Resume
Summary Statement is prepared
–SRA asks reviewers to modify critiques to reflect discussion
–SRA writes resume and summary of discussion in front
–Summary Statement (“Pink Sheets”) sent to applicant
![Page 29: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/29.jpg)
Final Score
• Average of all scores multiplied by 100
• Example:
–Average of raw scores from review panel = 1.88
–Final score = 188
![Page 30: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/30.jpg)
Percentiles
• Percentiles indicate your rank relative to other applications reviewed by group
• 0.1 (best) to 99.5 (worst) percentage of proposals receiving
better score during last year
• Example: Score: 188, Percentile: 11
![Page 31: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/31.jpg)
Applications Used in Percentile Conversion
• R01 reviewed at standing study section
–Percentile of score relative to all scores from current round plus last two rounds (1 year)
• R01 reviewed at special emphasis panel
–May be percentiled against distribution of all CSR scores
![Page 32: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/32.jpg)
http://www.niaid.nih.gov/ncn/grants/charts/flowchart_funding.htmhttp://www.niaid.nih.gov/ncn/grants/charts/flowchart_funding.htm
FundingDecisionFlowchart
FundingDecisionFlowchart
![Page 33: NIH Review Procedures](https://reader035.vdocuments.us/reader035/viewer/2022062314/568145aa550346895db2a095/html5/thumbnails/33.jpg)
Thank you