community conversations research grant review the review...•external reviewer/staff reviewer...
TRANSCRIPT
Community Conversations Research Grant Review
Live Orientation LaunchApril 26th 2018
Agenda
• Preparing for the Review
• Panel Information
• Review Participant Roles
• Review Form
• Sample Timeline
• Expectations for Comments and Clarification
• Next Steps
• Q&A
Preparing for the Review
• All Reviewer resources are posted at:
– https://cnsgov.sharepoint.com/sites/extranet/OGPO/2018communityconversations/SitePages/Home.aspx
– Reviewers will have four mandatory trainings (calls/webinars)
• Launching the Review (current training)
• Reviewing the Applications-Part I
• Reviewing the Applications-Part II
• Ensuring an Equitable Review (External Reviewers only)
• Conflicts of Interest LMS training (Staff Reviewers only)
– Read competition documents:
• Notice of Funding
– Application Instructions
– FAQs
Preparing for the Review
• Reading Materials
• Sample Application
• Review Materials:
– Review Form & Rubric
• Other reviewer resources and administrative forms on Reviewer Resource Webpage:
– Writing Meaningful Comments document
– Conflict of Interest and Participation Agreement Forms for External Reviewers
– Conflict of Interest Guidance and Participation Guidance for CNCS Staff Reviewers
Panel Introduction emails will be sent Monday, April 30th
Approximately 10 Applications per panel
1 External Reviewer
1 CNCS Staff Reviewer
1 Program Office Liaison
1 GARP Liaison
The GARP Liaisons and Program Office Liaisons will each have multiple panels.
Review Roles
• External Reviewer/Staff Reviewer
– Responsible for reading and assessing applications, engaging in panel discussion, addressing feedback, to complete high-quality review forms
• Program Office Liaison (POL)
– Serve as a resource to the panel on programmatic elements and answers content-specific questions
• GARP Liaison
– Manages review logistics for panels, ensures reviewers have completed requirements for orientation and access to materials, verifies final reviewer documents have been completed
Review Roles
• Review Coordinators
– Program Office Lead: Andrea Robles, R&E
• POLs will be contacted as needed if specific selection criteria questions arise, programmatic questions about ratings or aspects on a particular criteria
– Office of Grants Policy and Operations Lead: Nikki Hayden and Susan Cohn, OGPO
• Manages review process
• Confirm successful review form submission
Reviewer Responsibilities
• Review and score applications against selection
criteria and other considerations in the Notice
• Produce high quality assessments of the
strengths and weaknesses of the applications
• Help inform decision makers on quality of
applications to aid in their funding decisions
IRF Development Process
Use Review Form to guide you through the criteria for assessing the application
AFTER initial IRF is completed, then meet with panel partner for Panel Discussion
Return to IRF to revise as necessary. Save IRF using Naming Convention, then send to GARP Liaison
Receive GARP Liaison feedback, revise & resubmit (if necessary) until approval is received
OGPO informs the Reviewer when Approval is granted on each application
Panel Discussions
Consensus is not required of the panel discussion.
Reviewers are asked to engage in discussion
about the Criteria and consider the assessments
and finding of their fellow panel members. The
discussion should cover each of the relevant
elements of the application.
Panel Discussion Report
Panel Discussion Naming Convention
All review forms should follow the naming convention:
Legal Applicant Name. Application ID. PDR. Reviewer First Initial Last Name. Panel
Number.
Example:
UniversityofToronto.RX123456.PDR.ASmith.1
Review Form Naming Conventions
All review forms should follow the naming convention:
Legal Applicant Name. Application ID. IRF. Reviewer First Initial Last Name. Panel
Number.
Example:
UniversityofToronto.RX123456.IRF.ASmith.1
Submission of Review Materials
Both Panel Discussion Reports as well as Review Forms should be submitted to:
Required Timeline Dates
Review Begins:
Monday, April 30th
Review Ends:
Monday, May 21st
All IRFs must be approved as final by OGPO by 5:00pm EST
Week 1 Sample Timeline
• Monday, April 30th:
– All applications are received
– Reviewers have completed all orientations
– Begin reading applications for COI and follow up with GARP Liaison as needed
– Reach out via email to fellow panel member to schedule Panel Intro Call
Week 1 Cont. Sample Timeline
• Tuesday, May 1st:
–Hold Panel Intro Call to establish order of reading applications and introduce panel members (this may also be covered via email exchange)
–Begin reading and completing IRF for three applications
• By Fri, May 4th
–First Panel Discussion: discuss 1-3 applications
–Send first IRF’s to GARP Liaison
Week 2 Sample Timeline
• Wed, May 9th
– Receive feedback from GARP Liaison on first IRF
– Work with liaison to ensure first set IRF is approved
– Continue reading and reviewing applications 4-6
• Thu, May 10th:
– Second Panel Discussion: Discuss applications 4-6
• Fri, May 11th:
– Send IRFs for applications 4-6 to for GL review, and revise as necessary
– Continue reading and reviewing applications 7-10
Week 3 Sample Timeline
• Tue, May 15th:
– Receive feedback from GL on applications 4-6
– Final Panel Discussion for applications 7-10
• Wed, May 16th:
– Send remaining IRFs to GL for review
• Fri, May 18th
– Receive feedback from GL on 7-10
• Mon, May 21:
– All IRFs must be accepted as final by 5:00 pm EST
Tips for Producing High-Quality Review Forms
• Address only the Selection Criteria. Do not compare with other applications
• Address only the information within the application
• Use evaluative, grammatically correct language
• Comments and ratings should be aligned
Writing Strong Comments
• Comments should align with/support Ratings
– Fair or Does Not Meet should result in a Weakness comment
– Good or Excellent should result in a Strength comment
• Comments should relate directly to the selection criteria
• Do not just reiterate the criteria or repeat summary from the application, explain how the applicant’s response to the criteria is strong or weak
• Use complete sentences
• Significant Strengths and Weaknesses are used as feedback to applicants
Writing Items for Clarification
• Allows applicants to explain aspects of the application that were not clear, not intended as an option to provide additional information that should have been included in the initial application
• Allows applicants to adjust budget or other items that must be resolved prior to award
• Reviewers should always write Clarification as a clearly articulated question or statement requesting a response
Next Steps: • Complete all Orientation Sessions/Readings
by Mon, April 30th –Verify completion at [email protected]
• Receive Panel Assignment email – (Monday, April 30th)
• Panel Intro Call/email – (Tue, May 1st )
• Review Check-In Q&A Calls – Fri, May 4th 1:00p.m. Eastern
– Fri, May 11th 1:00 p.m. Eastern
Questions?