teacher quality partnership (tqp) grant program · 2019. 5. 25. · the teacher quality partnership...

TEACHER QUALITY PARTNERSHIP (TQP) GRANT PROGRAM

FY 2019 TQP GRANT COMPETITIONPEER REVIEW ORIENTATION

Agenda for Today• TQP Competition Overview

• Roles & Responsibilities of Key Peer Review Players

• Logistics Overview from Longevity Consulting, Inc.

• TQP Program Purpose and Eligibility Requirements

• FY 2019 TQP Program Priorities

• TQP Selection Criteria and Scoring

• Written Comments and Score Alignment

• Final Reminders

2

TQP COMPETITION OVERVIEW

3

TQP Grant Program

PurposeTo improve student achievement by improving the quality of new teachers.

Funding

October 1, 2019 4

TQP Project Start Date

$37 million (est.) to be obligated for fiscal year (FY) 2019. We plan to make 15-20 (est.) new awards.

LegislationThe Teacher Quality Partnership program statute is contained in Title II of the Higher Education Act, as amended on August 14, 2008, by the Higher Education Opportunity Act (Public Law 110-315) (HEA).

DATE TASK

May 23, 2019 Mandatory Peer Reviewer and Panel Monitor Orientation Webinar.

May 24, 2019 Peer Reviewers will receive electronic copies, via email, of all applications assigned to his/her panel. As this is a GREEN competition, paper copies will not be provided. Reviewers may also view and/or download applications from the G5 system. Peer reviewer materials will be available on the Longevity Consulting, Inc. TQP competition website and in the G5 system. Each peer reviewer has their own unique log in information for both the website and G5. Contact Longevity Consulting, Inc. if you have problems logging on to the website.

May 24 – June 2, 2019 Application Reading Period.

June 3-7, 2019 Application Panel Call Period. Peer reviewers must participate in all scheduled panel calls. Peer reviewers will use the Word Technical Review Form (TRF) to draft comments and scores for each application. Only final comments and scores will be entered into the G5 system.

June 10 -14, 2019 Final Comments and Scores entered into G5. Panel monitors, peer reviewers and Control Room staff will work together to ensure that edits are made to comments and scores as necessary, and are ultimately submitted as final in G5. Panel monitors will ensure that all final checklists for each application are turned in to the Competition Manager.

June 14, 2019 FINAL deadline for all comments and scores be submitted in the G5 system..

June 21, 2019 All honorarium forms must be returned to Longevity Consulting Inc.

5

Competition Timeline

TQP Panel Logistics

• XX TQP applications will be reviewed.• There will be XX applications per panel and 8

panels.Applications

6

Panelists

• There will be 3 peer reviewers and 1 panel monitor on each panel.

• Peer reviewers will review all parts of each application in depth.

• Panel monitors will review selected components of each application.

• To the extent possible, there will be a peer reviewer with experience in each area below on each panel:

• Teacher preparation and professional development;• Evaluation; and • Experience with STEM subjects.

TQP Panel Logistics

• 8:30 am – 11:30 am; or• 11:00 am – 2:00 pm; • 2:30 pm – 5:30 pm.

Panel Call Times

7

Panel Commitment Expectations

• Completion of panel calls and tier 1 clearance are PRIORITY for June 3-7, 2019

• Final comments and scores should be entered into G5 on a rolling basis during the week of June10-14, 2019

Accessing G5• Peer Reviewers will primarily only use G5 to enter FINAL

comments and scores.• Peer Reviewer may chose to download applications from G5, but

that is not required.• Peer Reviewers have their own unique log in for G5, be sure to

keep it handy.• Use the instructions provided in the “Accessing G5” handout

located on the TQP competition website provided by Longevity Consulting, Inc.

Help Desk Hours of Operation

8

Call Center Toll-Free Number: 1-888-336-89308:00 AM to 6:00 PM, Monday - Friday, EDT. On-line support is available by email at: [email protected]

• We are making every effort to reduce the amount of paper used during this peer review.

• Prior to the peer review, panelists will not receive paper copies of applications; rather electronic copies will be emailed to each panelist and will be available in G5.

• Panelists may download and/or print the documents at their own expense, as needed.

• Other competition resources can be found on the Longevity Consulting, Inc. TQP competition webpage and the TQP program webpage.

9

TQP Has Gone GREEN!

TQP Peer Review ResourcesTQP Program Website:

• FY 2019 TQP NIA• FY 2019 TQP Application

Instructions Package• FY 2019 TQP Application

Informational Webinars• FY 2019 TQP Application Checklist• TQP Match Documentation Letters• TQP Budget Narrative Template• Optional TQP Program Checklists

http://innovation.ed.gov/what-we-do/teacher-quality/teacher-quality-partnership/

Longevity Consulting, Inc. TQP Competition Website

• Panel Assignment Info• Peer Review Orientation PowerPoint• Reviewer Agreement (COI)• Peer Reviewer Guidebook• MS Word TRF• Absolute Priority Checklist• Applicant Eligibility Checklist• Honorarium Form• Accessing G5 TRFs

http://gms.lngvtycnslt.com/index.php/TQP-2019

10


http://gms.lngvtycnslt.com/index.php/TQP-2019

Conflicts of Interest (COI)Within 48 hours of receiving your applications, peer reviewers should ensure that you have no COI with any application assigned to your panel. If there is a COI, please alert your Panel Monitor IMMEDIATELY.A peer reviewer has a conflict of interest if:

• The reviewer has agreed to serve as an employee, advisor, contractor or consultant on a project for which funding is being sought in an application under review, or has been offered the opportunity to do so and has not yet accepted or declined the offer, based on whether a grant is awarded;

• The reviewer’s personal financial interests will be affected by the outcome of the competition;

• The reviewer helped prepare an application in the competition, even if the reviewer has no financial interest in the outcome of that application.

11

Financial Conflict of Interest The reviewer has a relationship with an entity or individual that has a financial interest in the outcome of the competition, including, but not limited to, the following: • The reviewer’s spouse, child, a member of household, or any relative with whom he

or she has a close relationship;• Any employer the reviewer has served within the last 12 months, a business partner,

an organization for which the reviewer has served as an officer, director, consultant, advisor, contractor, or trustee within the last 12 months, or an organization for which the reviewer serves as an active volunteer or participant;

• Any person or organization with whom the reviewer is seeking, or has an arrangement concerning, future employment;

• Any professional associate – including, but not limited to, any colleague, scientific mentor, or student – with whom the reviewer is currently conducting research or other professional activities or with whom he or she has conducted such activities within the last 12 months; or

• Any individual with whom the reviewer has, or has had, a personal relationship where the nature, duration, or recentness of that relationship would impair his or her ability to impartially review any application in the competition.

12

• Do not discuss information in applications outside the panel calls during or after the review process.

• Discuss them only with the panel members and only for the purpose of application review.

• Do not retain copies of documents or parts of documents related to this review.

• Do not contact the originator of the application concerning any aspects of its contents.

13

Confidentiality

ROLES AND RESPONSIBILITIES OF PEER REVIEW PLAYERS AND WORK FLOW

14

Key Peer Review Players

Peer Reviewers(PR)

• Verify eligibility requirements for each application• Use TRF to formulate comments and scores for each application on your panel• Discuss preliminary scores and comments with panel• Adjust comments and scores as necessary and provide them to PM for edits• Refer questions about applications or procedures to PM• Produce final comments and scores to be entered into G5

Panel Monitors(PM)

• Schedule and facilitate panel calls• Verify eligibility requirements for each application• Combine checklist results for each application and send to Competition Manager (CM)• Review and edit TRFs submitted by the peer reviewers• Answer procedural questions and refer complex issues to CM

Control Room(CR)

• Review and edit TRFs submitted by assigned PM• Oversee panel progress and ensure panels keep pace with reviews• Partner with PM and CM to ensure all FINAL comments and scores are entered into G5 by the

deadline

Key Peer Review Players (cont.)

Competition Manger

(CM)

• Oversee the overall peer review process• Address questions/concerns that have been elevated to the PM• Sign off on TRFs once they have been approved by CR• Work with Office of General Counsel (OGC) to get answers to legal questions

where necessary• Mia Howerton [email protected]

OGC Team• Provide answers to legal questions that have been elevated by the CM

Competition Logistics

Contractor

• Create panel assignments• Replace peer reviewers as needed• Process peer reviewer honorariums• Support the TQP team during the review• Longevity Consulting, Inc. [email protected]

mailto:[email protected]


Peer Reviewer Roles and Responsibilities

• Participate in all panel calls and adhere to the review timeline. • Use the Absolute Priority Checklist and TQP Applicant Eligibility Form

to verify if all statutory requirements have been met.• Read, evaluate, score and provide constructive comments for the

Selection Criteria, Competitive Preference Priority #1 and Invitational Priority, if applicable, for all assigned applications.

• Draw upon your expertise in identifying areas of need for new and effective interventions, programmatic design, and project execution, but do not introduce outside knowledge about any applicant.

• Make adjustments to comments and scores as necessary given the results of each panel call and edits from the PM and the CR staff.

• Upon approval and instruction, enter final comments only into the G5 system.

17

Panel Monitor Roles and Responsibilities

• Schedule and facilitate all panel calls and adhere to the review timeline.• Read application Abstracts for all assigned applications.• Use the Absolute Priority Checklist and TQP Applicant Eligibility Form to

verify if all statutory requirements have been met.• Verify GEPA and high-need LEA and school data provided by the applicant.• Review and score Competitive Preference Priority #2, if applicable, for all

assigned applications.• Serve as the first tier review of TRFs from your panel, making sure all

comments are clear, concise, constructive and reflective of the panel call discussion.

• Ensure that scores reflect the comments and are mathematically correct. • Ensure that comments between reviewers are not factually contradictory.• Partner with CR staff to ensure all edits to comments and scores are

completed and when final, are entered into G5.• Immediately send all panel questions to the CM.

18

TQP Program Checklists

TQP Absolute Priority Checklist

19

• Optional TQP Absolute Priority Checklist

TQP Program ChecklistsTQP Application Eligibility Form

20

• Optional TQP Eligible Partnership AND IHE Partner Verification Checklist

• Optional TQP General Application and Program Requirements

• Optional TQP High-Need LEA AND High-Need School Checklist

Panel Call Work Flow and Exchange

Phase 1 (Pre-Panel)• PR will confirm that there is no COI.• PRs will review all assigned

applications and verify eligibility information.

• PRs should use the MS Word TRF to draft comments and scores and be prepared to discuss applications on each call.

• PMs will review all assigned application Abstracts and verify eligibility information to include GEPA and high need data and CPP 2.

Phase 2 (Panel)• At the top of each call,

panelists will discuss and verify if all program requirements have been met.

• Panelists will discuss draft comments and scores assigned to each application.

• If necessary, PRs may change comments and scores based on the panel call outcomes.

• After each call, PRs will email draft comments and scores to the PM for Tier 1 review.

Phase 3 (Panel)• PMs will review draft

comments/scores using track changes to provide suggested edits.

• PMs will return comments to PRs to make corrections. PRs should also use track changes to show that edits have been made.

• Once all first round edits have been made and approved, PMs will forward comments to the CR staff for Tier 2 review.

Phase 3 (Post-Panel)• CR staff will review/edit TRFs

as part of the Tier 2 review.• CR staff will return comments to

PM and (if necessary) repeat steps in phases 2 and 3.

• Once approved, CR staff will forward TRFs to CM for final approval.

Phase 3 (Post-Panel)• CM will briefly scan comments

as part of Tier 3 review.• CM will approve comments for

final submission into G5.• CM will email CR staff and PM

to indicate when final comments may be entered into G5.

Phase 4 (Panel-Clearance)• Upon instruction from PM, PRs will

enter final comments and scores in G5

• PM will ensure that final comments and scores have been entered in G5.

• PM will email final two checklists for each application to the CM.

• PR will submit honorarium forms to Longevity Consulting.

Only final comments and scores should be entered into G5.

• Panel applications 1and 2• Receive/review TRFs for applications 1 and 2June 3• Provide PR feedback for applications 1 and 2• Panel applications 3 and 4• Receive/review TRFs for applications 3 and 4• Move TRFs for applications 1 and 2 to Control Room

June 4

June 5

June 6

June 7

Panel Calls – Week 1

22

• Provide PR feedback for applications 5 and 6• Panel applications 7 and 8• Receive/review TRFs for applications 7 and 8• Move TRFs for applications 5 and 6 to Control Room

• Provide PR feedback for applications 7 and 8• Move ALL TRFs to Control Room by 4:30 pm

• Provide PR feedback for applications 3 and 4• Panel applications 5 and 6• Receive/review TRFs for applications 5 and 6• Move TRFs for applications 3 and 4 to Control Room

Panel Calls – Week 2

23

• No panel calls, baring emergencies from Week 1• PMs will work closely with CR and CM to finalize comments

and scores• PMs will alert PRs of additional edits or when comments may

be entered in G5• PRs may enter final comments in G5 on a rolling basis during

this week• All Final comments and scores must be entered in G5 by

June 14, 2019

24

Three-Tier Peer Review Panel Monitor

(ED Panel Monitors)Tier 1

Control Room(Ashlee, Gillian, Mia)

Tier 2

Competition Manager(Mia Howerton)

Tier 3

• The TRF is completely filled out• Comments are clearly written in

paragraph or bullet form• Comments reflect the selection

criteria and sub-factors• Comments are clear and are

supported by evidence from the application

• Comments are not factually contradictory across the panel

• Scores given are supported by comments that reflect the score

• Scores are numerically accurate• Strength and weakness comments are

included where appropriate• Statements are supported with page

numbers• ZERO scores are supported by all

three reviewers• Grammar and spelling is accurate• Check for large variances in scores

(20%)

• The TRF is completely filled out

• Comments are clearly written in paragraph or bullet form

• The scores given are supported by comments that reflect the score

• Strength and weakness comments have been included where appropriate

• Comments are not factually contradictory across the panel

• Check for large variance in scores (20%)

• Scan TRF for overall completion and score comparisons

• Check for large variance in scores (20%)

• Provide final approval to PM and CR staff when PR may enter comments and scores into G5.

Replacement of ReviewersIf you feel that you cannot perform the duties as described, please contact your Panel Monitor or the TQP Competition Manager ASAP.

Reasons reviewers have self-selected to withdraw from the process include:

• Personal Emergencies

• Work related issues that prohibit full participation

• Family obligations

On rare occasions, the Department may find it necessary to replace a reviewer after the start of the review process if:

• There are conflict of interest issues

• The reviewer is not performing to expectations

25

QUESTION AND ANSWERSESSION

26

LOGISTICS OVERVIEW FROM LONGEVITY CONSULTING, INC.

27

Copyright © 2017 Longevity Consulting LLC. All rights reserved.

Longevity Contact:

Karen Gibson-Serrette, Project Manager

301-379-5817

[email protected]

Longevity Email:

[email protected]

Longevity Consulting, Inc.


Longevity’s Peer Reviewer Website

This website serves as a resource for your peer review needs. Background materials on this competition, resources for G5 access, and forms required for submission can all be found here along with a peer review schedule to keep you updated on milestones.

Website Link Hyperlink from your confirmation email

What you Need to Login All found in your confirmation email

First and Last Name Email Address Confirmation Number



Peer Review Schedule Due Dates of Competition Milestones

Materials & Forms Background of Competition Orientation Webinar Information Reviewer Guidance Forms

General Information Overview for New Reviewers Navigating the G5 System Bulletin Site Map

Contact Information Email Form and TL Staff

My Panel Panel Monitor Contact Information Panel Slot Availability Link Panel Slot Time Assigned Applications Panel Member Contact Information

My Registration Confirmation Number

Each page provides an abundance of information categorized in different tabs.


Once you have completed your comments and scores in G5, don’t forget to submit the two (2) documents needed to receive your honorarium. Signed ED Form 5249 – located on the Forms Page of the Reviewer Website Completed and Signed Honorarium Form– located on the Forms Page of the Reviewer Website




If you have questions for Longevity, please email them toKaren Gibson-Serrette, Project [email protected]

TQP PROGRAM PURPOSE AND ELIGIBILITY REQUIREMENTS

33

TQP Program Purpose

34

To improve student achievement; improve the quality of prospective and new teachers by improving the preparation of prospective teachers and enhancing

professional development activities for new teachers...



Who is Eligible?

35

Applicants MUST be an “eligible partnership” as defined in HEA section 200(6):

All five (5) entities MUST be included as part of the “eligible partnership”

High-Need LEA(s)

High-Need School(s) served by the High-

Need LEA(s)

Partner IHE

College or School of Arts and Sciences within the partner

IHE

College or School or Program of

Education within the partner IHE

Optional Entities for the Eligible Partnership

36

(B) MAY include any of the following:(i) The Governor of the State.(ii) The State educational agency.(iii) The State board of education.(iv) The State agency for higher education.(v) A business.(vi) A public or private nonprofit educational organization.(vii) An educational service agency.

Optional Entities for the Eligible Partnership (cont.)

37

(viii) A teacher organization.(ix) A high-performing local educational agency, or a consortium of such local educational agencies, that can serve as a resource to the partnership.(x) A charter school (as defined in section 7221i of this title).(xi) A school or department within the partner institution that focuses on psychology and human development.(xii) A school or department within the partner institution with comparable expertise in the disciplines of teaching, learning, and child and adolescent development.(xiii) An entity operating a program that provides alternative routes to State certification of teachers.

Eligibility Components

38

High-Need LEA

Poverty Data/Rural

StatusTeacher

Need

Documentation of these eligibly requirements must be included in the application at the time of submission.

High-Need Schools

(within the High-Need LEA)

Free and Reduced Price Lunch

(FPRL)

Poverty Data/Rural Status

Census data (or other comparable data)

showing LEA serves 20% or more children

from low income families

Census data (or other comparable data)

showing LEA serves 10,000 or more children from low income families

Eligible for ESEA’sSmall, Rural School

Achievement Program (20 USC §7345(b))

Eligible for ESEA’s Rural and Low-Income School

Program (20 USC §7351(b))

Data MUST be documented in ONE of four ways.

Teacher Need Data

Data MUST be documented in ONE of these three ways.

(1) High percentage of teachers not teaching in

the academic subject areas or grade levels in which the teachers were

trained to teach

(2) High teacher turnover rate

(3) High percentage of teachers with emergency, provisional, or temporary certification or licensure

High-Need School Data

Data MUST be documented in ONE of three ways.

In highest quartile of schools in LEA using FRPL

data

Elementary Schools - 60% or more of students

eligible for FRPL

Not an Elementary School - 45% or more of

students eligible for FRPL

TQP General Program RequirementsRequirement Area Location in Statute Location in Project

Narrative

Needs Assessment 202(b)(1) Appendix C

Description of Project 202(b)(2,3,6(A) & (F-J)) Project Design

Description of coordination strategies and alignment with state and student academic achievement standards

202(b)(4(A-B),6(B-C) & (E))

Project Design,

Assessment of the resources available

202(b)(5) Management Plan, Adequacy of Resources, Budget Narrative

Description of Evaluation Plan 202(b)(6(D) & (K)) Evaluation Plan

42

100% Non-Federal Matching Requirement

– Required 100% match funding for each year of the TQP grant.

– Matching funds may be cash or in-kind donations.– Applicant should include documentation that all or

part of the match has been met.– If applicant is unable to meet the 100% fund

match they may request a full or partial waiver for the match requirement for Year 1.

– The Department does not anticipate approving any waivers to the 100% non-Federal matching requirement.

43


44

FY 2019 TQP PROGRAM PRIORITIES

45

Applicants MUST address the

Absolute Priority

Partnership Grants for the

Establishment of Effective Teaching

Residency Programs

(Residency Model)

Projects designed to improve student achievement or

other educational outcomes in

computer science (0-5 points)

Applicants may address any of these

Competitive Preference Priorities

FY 19 TQP Program Priorities

46

Novice Applicants (5 points)

Spurring Investment in Opportunity

Zones(No additional

points)

Applicants may address this Invitational

Priority

Absolute Priority

47

• Applicants should have read the TQP NIA for full Absolute Priority language.

• Applicants must address ALL Absolute Priority requirements or may be deemed ineligible.

• Panelists will use the Optional Absolute Priority Checklist to verify if all Absolute Priority requirements have been met by each applicant.

48

Absolute Priority: Residency Model Highlights

Residency models establish/design new teaching residency programs

Residency models must have a rigorous selection criteria for residents and mentor teachers

Residency models shall provide a one-year Living Wage Stipend using federal funds

Residency model graduates must teach for 3 years in the partner high-need LEA or repay the living wage stipend

Residency models must provide a 2-year Induction Program Residency models must provide training and support for teaching

residents Residency models must offer an 18-month Master’s degree and

teacher certification program

(See Section 202(e) of the HEA for full Absolute Priority language)

49

Residency Model – Clarifications The Residency model is for graduate students only, no

undergraduates. The Residency model supports those who will be first year

teachers, not existing teachers. Residents must receive a state teaching certification AND a

Master of Teaching degree. The MAT program must have an 18-month design. Existing residency programs may be funded, however they

must meet all the Absolute Priority requirements. Tuition and tuition reductions are not allowable cost under

the Residency model and cannot be used as match. Living wage stipends are required, not optional. Residency programs are responsible to track whether a

resident project meets the three year service requirement. If a resident choses to use the Living Wage Stipend to pay

their tuition, they must do so on their own.

Competitive Preference Priorities (CPPs)

50

The CPPs were optional; applicants were not required to address these priorities.

Applicants were required to clearly identify if they addressed one or both CPPs and where the responses can be found in the application.

The CPPs are worth up to an additional 10 points. Peer Reviewers will score CPP 1. Panel Monitors will score CPP 2.

51

Competitive Preference Priority 1(0-5 points)

Projects designed to improve student achievement or other educational outcomes in computer science by increasing the number of educators adequately prepared to deliver rigorous instruction in STEM fields, including computer science, through recruitment, evidence-based professional development strategies for current STEM educators, or evidence-based retraining strategies for current educators seeking to transition from other subjects to STEM fields.

52

Competitive Preference Priority 1How does an applicant, in addressing CPP 1, demonstrate that its proposed strategy to improve student achievement or other educational outcomes in computer science is evidence-based?Applicants must demonstrate, in order to meet CPP 1, that their proposed professional development or retraining strategies are “Evidence-Based,” as defined in 34 CFR §77.1. Applicants may do so by:

Submitting a “Logic Model” (as defined in 34 CFR §77.1) that (1) identifies the STEM professional development or retraining strategy of the project; and (2) is informed by research or evaluation findings that suggest the project component is likely to improve “Relevant Outcomes” as defined in 34 CFR §77.1.

Submitting a citation of a study that is (1) focused on a STEM-focused professional development or retraining strategies, (2) relevant to the proposed project, and (3) meets at least the design standards set forth in the “Promising Evidence” definition in 34 CFR §77.1.

53

Competitive Preference Priority 2 (5 pts)Novice Applicants

Projects submitted by applicants that meet the definition of novice applicant at the time of they submit their application.

• Has never received a grant or sub-grant under the TQP program; and

• Has never been a member of a group application (i.e., in a TQP eligible partnership); and

• Has not had an active discretionary grant from the Federal government in the five years before the deadline date for applications under the program.

Invitational Priority (IP)

54

The IP was optional; applicants were not required to address this priority.

Applicants were required to clearly identify if they addressed the IP and where the responses can be found in the application.

The IP is not worth any additional points.

55

Invitational Priority

Spurring Investment in Opportunity Zones.(1) Propose to serve children or students who reside, or attend TQP

project schools, in a qualified opportunity zone as designated by the Secretary of the Treasury under section 1400Z-1 of the Internal Revenue Code, as amended by the Tax Cuts and Jobs Act (Pub. L. 115-97). OR

(2) Demonstrate in its application that it has received or will receive financial assistance from a qualified opportunity fund under section 1400Z-2 of the Internal Revenue Code, as amended by the Tax Cuts and Jobs Act, for a purpose directly related to its proposed project. In addressing this priority, an applicant must identify the qualified opportunity fund from which it has received or will receive financial assistance.

56

What is an Opportunity Zone?

An Opportunity Zone is an economically-distressed community where new investments, under certain conditions, may be eligible for

preferential tax treatment. Opportunity Zones are designated by the state and certified by the IRS.

Roughly 8,700 areas in all 50 states have been designated as Opportunity Zones.


57

FY 2019 TQP SELECTION CRITERIA AND SCORING

58

2019 TQP Selection CriteriaAbsolute Priority should be addressed in response to the Selection Criteria.

Selection Criteria Maximum Points (100)

Quality of Project Design Up to 40 points

Adequacy of Resources Up to 20 points

Quality of Management Plan Up to 20 points

Quality of Evaluation Plan Up to 20 points

Competitive Preference Priorities (OPTIONAL)

Up to 10 points

59

Balance of Costs with Outcomes of

Project

Selection Criterion: Quality of the Project Design

Does our Logic Model reflect research and

evaluation that support your project design

being effective?

1. The extent to which the proposed project demonstrates a rationale.

2. The extent to which goals, objectives, and outcomes to be achieved by the proposed project are clearly specified and measureable.

60

Are your project objectives S.M.A.R.T?

What is a Logic Model?A Logic Model is a visual diagram that illustrates how the project will work. Logic models communicate a project’s vision, objectives, goals to their internal team and stakeholders. Logic Models can also be used in program planning, implementation, evaluation, and communication.

Logic Models reflect the goals and activities in the grantee’s application.

Logic Models identify the grantee’s projects and each project’s expected outcome.

Logic Models identify sustainable changes for the grantee. Logic models contain appropriate information to serve as a

communication tool to grantee stakeholders. 61

Balance of Costs with Outcomes of

Project

Selection Criterion: Quality of the Project Design

Can this program’s impact be sustained

after the Federal project period?

Have all the absolute priority requirements been addressed in an

exceptional and innovative way?

3. The extent to which the proposed project is designed to build capacity and yield results that will extend beyond the period of Federal financial assistance.

4. The extent to which the proposed project represents an exceptional approach to the priority or priorities established for this competition.

62

Selection Criterion: Adequacy of Resources

1.The adequacy of support, including facilities, equipment, supplies, and other resources, from the applicant organization or the lead applicant organization.

2. The relevance and demonstrated commitment of each partner in the proposed project to the implementation and success of the project.

Are the lead agency’s resources and

support mechanisms sufficient to ensure project success?

Do you have full support from project partners to make the proposed project a

success?

63

NOTE: Adequacy of Resources

The NIA selection criteria includes factors such as “the adequacy of support, including facilities, equipment, supplies, and other resources, from

the applicant organization or the lead applicant organization” and “the relevance and

demonstrated commitment of each partner in the proposed project to the implementation and success of the project,” which may include a

consideration of demonstrated matching support.

64

Selection Criterion: Quality of the Management Plan

Is there a viable plan to carry out

the proposed project?

1. The adequacy of the management plan to achieve the objectives of the proposed project on time and within budget, including clearly defined responsibilities, timelines, and milestones for accomplishing project tasks.

65

1. The extent to which the methods of evaluation will provide valid and reliable performance data on relevant outcomes.

2. The extent to which the methods of evaluation are thorough, feasible, and appropriate to the goals, objectives, and outcomes of the proposed project.

Selection Criterion:Quality of Project Evaluation

Are key questions being asked that will

provide data to support project outcomes?

What data will be generated to help

determine if the project has met its goals?

66

• PR: No outside factors, such as personal knowledge of past performance, perceived relative quality based on reading other applications, or subjective judgments about what an application should contain may be considered .

• PM: Will not provide reviewers with evaluative guidance or feedback on the substance of an application, on reviewer scores, or on the content of reviewer comments beyond what is necessary to ensure that reviewer scores are mathematically correct and reviewer comments clearly address the scoring criteria and adequately justify reviewer scores.

Selection Criteria Reminder

67


68

WRITING COMMENTS AND SCORE ALIGNMENT

69

Scoring TQP ApplicationsPeer reviewers must provide both numeric scores and written comments using the MS Word TRF. Once comments and scores are approved, the panel monitor will provide

instructions for when to enter final comments and scores into G5.

Peer reviewers should only enter FINALcomments and scores into G5.

70

TQP Application Numeric ScoresNumeric scores indicate how well the applicant responded to the selection criteria.

• How well not only means how well the application is written, but also the extent to with the each criterion is addressed and with what level of quality.

• Peer reviewers should use the entire range of points for each criterion to differentiate application quality.

• Numeric scores are used to rank order applications when making decisions about funding applications.

71

Maximum Point Value

General Quality of Applicant’s ResponseNot

AddressedPoorly

DevelopedAdequately Developed

Well-Developed

Fully Developed

40 01-9 10-19 20-29 30-40

20 01-5 6-10 11-15 16-20

Sample Scoring Rubric

72

Shown below are suggested point ranges for an evaluation of applicant responses to a selection criterion that are fully developed, well developed, adequately developed, poorly developed and not addressed.

TQP Application Written CommentsWritten comments provide justification for the numeric scores and offer feedback to applicants.

• If full points are awarded to a criterion, all comments should reflect strengths and no weaknesses.

• If partial points are awarded to a criterion, there should be comments that highlight both strengths and weaknesses of the submitted application.

• If a score of ZERO is awarded to a criterion, this means the criterion was not addressed at all in the project narrative and all three reviewers must agree that a score of zero is warranted and awarded by all reviewers.

73

Writing Comments: The BasicsEach comment should: Make clear, evaluative statements about the substance of the

selection criterion being discussed; Substantiate statements using evidence from the application

narrative, appendices, and/or budgets without simply summarizing the application;

Use bullets to organize related evaluative statements clearly; Draw clear conclusions that are consistent with your evaluative

statements and match the score you assigned; and Use the selection criterion language and the scoring guidance as

your ultimate guidelines– make sure your scores and comments are aligned.

74

Writing Comments: Content

Explain why you reached the score you did. Refer to specific information in the application that helped

you reach your score (e.g., use “e” page numbers). Evaluate what the application says; DO NOT restate what

the applicant has written. If information is missing from the application, clearly indicate

this in your comments. Comments should reflect your best judgment based on the

information the applicant has presented and your professional expertise.

You comments and scores should not be based on outside research or sources.

75

Writing Comments: Style

Use simple, declarative sentences whenever possible.Use statements; DO NOT ask questions of the

applicant. Be professional, tactful, and constructive.Do not use statements that infer personal bias, such as

“I feel,” or “I think.” Avoid providing advice on what the applicant should

have done; focus on what they presented. Cite the application e-page numbers in your comments.Use bullet points when writing comments.

76

Sample Comment

“The application proposes to create a professional development program for teachers. It’s an excellent idea.”

77

Revised Sample Comment“One of the strengths of the proposed professional development program is that it gives teachers direct practice in using the newly created online repository of student assignments that can be used “in the moment” when classroom assessments identify a learning deficiency. As the application demonstrates on p. 5, this technology-based approach aligns with research showing that learning deficits can be remedied if addressed immediately.” This reviewer believes this is an excellent idea!

Why Is It Better?

It clearly explains why the reviewer believes this to be a quality program and includes specific references to the proposal.

78

Writing Comment Checklist Do my comment evaluate the application rather than just

summarize it? Have I provided evidence and included e-page numbers in my

comments. Do my comment address the selection criteria? Do my comment support the score? Did I use statements and not questions in my comments? Are my comments in bullet format, not paragraphs? Are my comments free from references to outside information? Are my comment free of “I” statements (I think, I feel etc.)?

If you answered “No” to any of the above,

your comments may needs to be REVISED!79

QUESTIONS AND ANSWERSSESSION

80

FINAL REMINDERS

81

Peer Reviewer Reminders• This TQP Competition Review MUST be completed in two weeks

(June 3-14, 2019).• Be prepared to review the first two applications on June 3, 2019.• Review the TQP NIA, Application Package, TQP FAQs and other

resources listed on the TQP program and TQP competition websites before you review applications.

• Identify any COI within 48 hours of receiving your assigned applications. Notify Longevity and/or Panel Monitor IMMEDIATELY if there are any COI.

• Maintain frequent communication with your assigned Panel Monitor.

82

Peer Reviewer Reminders• Review all parts of each assigned application and provide

aligned comments and scores that reflect how well or how poorly applicants responded to the criteria.

• Use optional TQP program checklists to verify statutory program requirements.

• All final comments and scores should be entered in the G5 system no later than June 14.

• All signed Honorarium Forms must be submitted to Longevity Consulting, Inc. no later than June 21.

• Peer Reviewers will not be paid until the review is complete and all Honorarium Forms have been submitted to Longevity Consulting, Inc.

83

Panel Monitor Reminders• This TQP Competition Review MUST be completed in two weeks

(June 3-14, 2019).• Establish panel call schedule, facilitate panel calls and maintain

frequent communication with your panel. • Be prepared to review the first two applications on June 3, 2019.• Review application abstracts and use optional TQP program

checklists to verify statutory program requirements.• Specifically verify GEPA, high need data, and CPP 2 for each

application, if applicable.• Serve as Tier 1 reviewer for TQP TRFs.• Instruct peer reviewers when final comments may be entered into

G5.• Complete final panel checklists and return them to the Competition

Manager by June 14, 2019.84

Opting Out

Based upon the information provided today, if you

feel that you cannot perform the duties as described,

please contact the TQP Competition Manager at

[email protected] immediately following this

presentation to let her know.

85



86

THANK YOUFOR BEING A VALUED

TQP PEER REVIEW PLAYER!

teacher quality partnership (tqp) grant program · 2019. 5. 25. · the teacher quality partnership...

Documents