Day of Service Grant Review: Martin Luther King Jr Day of Service & September 11 Day of Service and Remembrance

Live Orientation LaunchMarch 6, 2017

Agenda

• Preparing for the Review

• Panel Information

• Review Participant Roles

• Review Form Development Process

• Sample Timeline

• Expectations for Comments and Clarification

• Next Steps

Preparing for the Review

• All Reviewer resources are posted at:https://www.nationalservice.gov/DayofServiceReviewerPage2017

– Listen to four pre-recorded Orientations (calls/webinars)

• Selection Criteria: Program Design

• Selection Criteria: Organizational Capability; Cost-Effectiveness & Budget Adequacy

• Using the Individual Review Form (IRF)

• Ensuring an Equitable Review (External Reviewers only)

– Conflicts of Interest LMS training (Staff Reviewers only)

– Read competition documents:

• Notice of Funding, including Performance Measure Instructions

– Application Instructions

– FAQs

– Competition TA call

Preparing for the Review

• Review Materials:

– Review Form & Rubric

• Other reviewer resources and administrative forms on Reviewer Resource Webpage:

– Writing Meaningful Comments document

– Conflict of Interest and Participation Agreement Forms for External Reviewers

– Conflict of Interest Guidance and Participation Guidance for CNCS Staff Reviewers

Panel Composition

Panel Introduction emails will be sent Tues, March 7th

4 - 5 Applications per panel

1 External Reviewer

1 CNCS Staff Reviewer

1 Program Office Liaison (POL)

1 GARP Liaison

The POLs and GARP Liaisons will each have one to two panels.

Review Roles

• External Reviewer/Staff Reviewer– Responsible for reading and assessing applications,

engaging in panel discussion, and addressing POL feedback, to complete high-quality IRFs

• Program Office Liaison (POL)– Reviews and provides feedback on IRFs, until final approval

– Serve as a resource to the panel on programmatic elements: answers questions on selection criteria, serves as liaison to program office when necessary

• GARP Liaison– Manages review logistics for panels, ensures reviewers have

completed requirements for orientation and access to materials, verifies final reviewer documents have been completed

Review Roles

• Review Coordinators

– Program Office Lead: Patti Stengel, CPO Office

• POLs will contact Patti as needed if specific selection criteria questions arise, programmatic questions about ratings or aspects on a particular criteria

– Office of Grants Policy and Operations Lead: Femi Estrada-Petersen, OGPO

• Manages review process

IRF Development Process

Use Review Form to guide you through the criteria for assessing the application

AFTER initial IRF is completed, then meet with panel partner for Panel Discussion

Return to IRF to revise as necessary. Save IRF using Naming Convention, then send to POL for POL review

Receive POL feedback, then revise & resubmit until POL approval is received

POL informs the Reviewer and GARP Liaison when POL Approval is granted on each application

POL Role in IRF Review

• POLs will review IRFs and provide feedback to Reviewers to ensure that they are high-quality and address CNCS Standards

• POLs will check to ensure:– Comments are evaluative in nature, and there are an appropriate level

of comments for each section

– Comments are clearly articulated, address the selection criteria, and support the Ratings

– Correct phrasing of clarification items

• If the IRF includes no clarification, POL will reach out to confirm that this is correct

– General quality issues: Spelling and grammatical errors, completion

• POLs are a 2nd level check. It is Reviewer’s responsibility to ensuring your IRF is high-quality prior to sending to your POL

• Once approved, POL sends the approved IRF to PeerReviewers mailbox, copying the Reviewer & GARP Liaison

POL Role

• POLs will need one business day to review and provide feedback on IRFs. (Keep in mind that some POLs have more than one panel.)

• Ideally one review will be enough for each IRF, but the POL may ask for additional feedback reviews if the IRF still needs work

• If there is a misunderstanding of criteria or with quality of multiple IRFs, the POL may arrange to speak with the panel

• POL may consider joining the Panel Intro Call but will not join discussion calls

• Once review schedule is established, panel should share schedule with Liaisons

Required Timeline Dates (all times are Eastern)

• By no later than 5:00 pm Mon, March 13:

– Hold first panel discussion call (& submit IRF for POL review)

• By no later than 5:00 pm Tues, March 14:

– Have at least one IRF approved by your POL

• By no later than Noon Fri, March 17:

– Reviewers must have sent ALL draft IRFs for POL Review

• By no later than 5:00 pm Mon, March 20:

– All IRFs must be approved as final and submitted to PeerReviewers mailbox

Sample Timeline

• Wed, March 8:

– All applications are received

– Reviewers have completed all orientations

– Begin reading applications for COI and follow up with GARP Liaison as needed

– Reach out via email to fellow panel member to schedule Panel Intro Call

• Thurs, March 9:

– Hold Panel Intro Call to establish order of reading applications and introduce panel members (this may also be covered via email exchange)

– Begin reading and completing IRF for first application

Sample Timeline

• Fri, March 10:

– First Panel Discussion: discuss 1 application

– Send first IRF to POL

• Mon, March 13:

– Receive feedback from POL on first IRF

– Work with POL to ensure first IRF is approved

– Continue reading and reviewing applications 2-3

• Tues, March 14:

– Second Panel Discussion: Discuss applications 2-3

Sample Timeline

• Wed, March 15:

– Send IRFs for applications 2-3 to for POL review , and revise as necessary

– Continue reading and reviewing applications 4-5

• Thurs, March 16:

– Final Panel Discussion for applications 4-5

• Fri, March 17:

– Send remaining IRFs to POL for review

• Mon, March 20:

– All IRFs must be final by 5:00 pm EST

Writing Strong Comments

• Comments are used to inform the decision –making process, and as feedback to applicants

• Write at least one strength/weakness for each comment section

• Comments should align with/support Ratings

– Fair or Does Not Meet should result in a Weakness comment

– Good or Excellent should result in a Strength comment

• Comments sections should not be left blank

– If all criteria are assessed as Fair or Does Not Meet, then enter N/A in the Strengths section

– If all criteria are assessed as Good or Excellent, you may enter N/A in the Weaknesses

Writing Strong Comments

• Comments should relate directly to a specific criteria

• Do not just reiterate the criteria or repeat summary from the application, explain how the applicant’s response to the criteria is strong or weak

• Use complete sentences

• List each comment individually

• Significant Strengths and Weaknesses are used as feedback to applicants

• Examples of poorly written comments: – The second paragraph of page 12 confused me (does not

clearly say what is confusing)

– The partner engagement plan wasn’t very good (value judgement without specific information)

– Not all sites are listed (incomplete sentence and not detailed)

– What is meant by…?

• Examples of High-Quality comments– The subgrantee selection plan is detailed and clearly

describes the plan from recruitment of applicants to the weekly technical assistance webinars that will be provided in the month prior to MLK Day.

– The description of sites is incomplete, including only two of the required ten service locations. It was not clear if the other locations were identified, or if the applicant only planned two sites.

Writing Strong Comments

Writing Items for Clarification

• Allows applicants to explain aspects of the application that were not clear, not intended as an option to provide additional information that should have been included in the initial application

• Allows applicants to adjust budget or performance measure items that must be resolved prior to award

• Reviewers should always write Clarification as a clearly articulated question or statement requesting a response

Examples of Clarification

• The subgranting plan does not clearly show in the budget how many mini grants and how many larger subgrants will be given. Please adjust the budget to clearly show the correct breakdown of mini grants and large subgrants.

• The relationship between the Program Design and the current AmeriCorps program is unclear. What is this relationship?

• The O1 Economic Opportunity performance measure selected does not clearly describe the financial literacy services that will be provided. This description must be added to the O1 output description.

Next Steps: • Complete all Orientation Sessions

– Verify Completion of each session by sending one thing you learned or one question that you have about what was covered

– Send verification to PeerReviewers@cns.gov

• Receive Panel Assignment email – (Tues, Mar 7th)

• Panel Intro Call/email – (Wed, Mar 8th – Thurs, Mar 9th)

• Review Check-In Call – (Thurs, March 9th 1:00p.m. Eastern)

– Call is open Q&A, CNCS will ensure that all Reviewers feel prepared to proceed with the review and are ready to go!