americorps technical review cover sheet...dec 21, 2020 · section b program design (0-30 points...

1

2021 New York State AmeriCorps Competitive Application Review Process

REVIEWER WORKSHEET

(The individual reviewer worksheet does not determine final consensus scoring)

Proposal Name:

Reviewer Name (Print):

Section Totals

Section A Executive Summary: (Required) SUBTOTAL: YES / NO

Section B Program Design: Theory of Change and Logic Model (0-24) ______ + EVIDENCE BASE + Notice Priority (0) + Member Experience (0-6) ______

Note: Evidence Base (20 points) will be reviewed separately by OCFS Strategic Planning and Policy Development staff on 2021 Evidence Review Tool = SUBTOTAL: ________ (0-30)

Section C Organizational Capability: Organization Background and Staffing (0-9) ______ + Compliance and Accountability (0-8) ______ + Culture that Values Learning (0-4) ______ +

Member Supervision (0-4) ______ = SUBTOTAL: ________(0-25)

Section D Cost Effectiveness & Budget Adequacy: Cost Effectiveness and Budget Adequacy (0-25)

= SUBTOTAL: ________ (0-25)

Section E Evaluation Plan: (Required for recompeting grantees only) YES / NO

APPLICATION SCORE: ________(0-80)

2

Section A: Executive Summary Executive Summary (No points) Comments:

• Did the applicant provide an ExecutiveSummary? Yes No

Section B: Program Design (8 subsections)1.Theory of Change and Logic Model (0-24 points) Comments:

The Theory of Change Shall address:

• Did the applicant clearly describe that theproposed intervention is responsive to theidentified community problem? Yes No

• Did the applicant clearly describe theproposed intervention is clearly articulatedincluding the design, dosage, targetpopulation, and roles of AmeriCorpsmembers and (if applicable) leveragedvolunteers?

Yes No

• Did the applicant clearly describe how theintervention is likely to lead to the outcomesidentified in the applicant’s theory ofchange?

Yes No

3

Section B: Program Design (CONTINUED)

• Did the applicant clearly describe how theexpected outcomes articulated in theapplication narrative and logic modelrepresent meaningful progress inaddressing the community problemidentified by the applicant?

Yes No

• Did the applicant clearly describe therationale for utilizing AmeriCorps membersto deliver the intervention(s) is reasonable?

Yes No

• Did the applicant clearly describe theservice role of AmeriCorps members willproduce significant contributions to existingefforts to address the stated problem?

Yes No

Note: The logic model is a visual representation of the applicant’s theory of change. Programs may include short, medium or long-term outcomes in the logic model. Applicants are not required to measure all components of their theory of change. The applicant’s performance measures should be consistent with the program’s theory of change and should represent significant program activities.

In the application narrative, applicants should discuss their rationale for setting output and outcome targets for their performance measures.

Rationales and justifications should be informed by the organization’s performance data (e.g., program data observed over time that suggests targets are reasonable), relevant research (e.g., targets documented by organizations running similar programs with similar populations), or prior program evaluation findings.

Applicants with multiple interventions should complete one Logic Model chart which incorporates each intervention. Logic model content that exceeds three pages will not be reviewed.

4

Section B: Program Design (CONTINUED) The Logic Model shall depict:

• Did the applicant provide a summary of thecommunity problem. Yes No

• The inputs or resources that are necessaryto deliver the intervention, including but notlimited to:o Locations or sites in which members

will provide serviceso Number of AmeriCorps members that

will deliver the intervention

Yes No

• The core activities that define theintervention or program model thatmembers will implement or deliver,including:o The duration of the intervention (e.g.,

the total number of weeks, sessions ormonths of the intervention).

o The dosage of the intervention (e.g.,the number of hours per session orsessions per week.)

o The target population for theintervention (e.g., disconnected youth,third graders at a certain readingproficiency level).

Yes No

• The measurable outputs that result fromdelivering the intervention (i.e. number ofbeneficiaries served types and number ofactivities conducted.) If applicable, identifywhich National Performance Measures willbe used as output indicators.

Yes No

5

• Outcomes that demonstrate changes inknowledge/skill, attitude, behavior, orcondition that occur as a result of theintervention. If applicable, identify whichNational Performance Measures will beused as outcome indicators.

Yes No

Theory of Change and Logic Model (0-24 points) Total Points Awarded: _________

Section B: Program Design (CONTINUED) 2. Evidence BaseNote: Evidence Base (20 points) will be reviewed separately by OCFS Strategic Planning and Policy Development staff on 2021 Evidence Review Tool

3. Notice Priority (0 points) Comments: • Did the applicant clearly describe how its

proposed program is within one or more of the2021 AmeriCorps Funding Priorities section ofthe 2021 Notice of Funding Availability andmore fully described in the 2021 MandatorySupplemental Guidance and the proposedprogram meets all of the requirements detailedin the Funding Priorities section and in the2021 Mandatory Supplemental Guidance.

Yes No

Notice Priority (0 points allowable)

4. Member Experience (0-6 points) Comments:

• Did the applicant clearly describe how theAmeriCorps members will gain skills as aresult of their training and service that canbe utilized and will be valued by futureemployers after their service term iscompleted?

No Yes

6

• Did the applicant clearly describe how theprogram will recruit AmeriCorps membersfrom the geographic or demographiccommunities in which the programoperates?

Yes No

• Did the applicant clearly describe how theapplicant will foster and inclusive serviceculture where different backgrounds,talents, and capabilities are welcomed andleveraged for learning and effective servicedelivery?

Yes No

Member Experience (0-6 points) Total Points Awarded: _________

Section B Program Design (0-30 points allowable for all 5 subsections) TOTAL POINTS AWARD: ________

Section C: Organizational Capacity (4 subsections) 1. Organizational Background and Staffing (0-9) Comments:

• Did the applicant clearly describe the roles,responsibilities, and structure of the staff thatwill be implementing the AmeriCorps programas well as providing oversight and monitoringfor the program?

Yes No

Organizational Background and Staffing (0-9 allowable) Total Points Awarded: _________ 2. Compliance and Accountability (0-8) Comments: • Did the applicant clearly describe a monitoring

and oversight plan to prevent and detect non-compliance and enforce compliance withAmeriCorps rules and regulations includingthose related to prohibited and unallowableactivities and criminal history checks atgrantee, subgrantee (if applicable), andservice site locations?

Yes No

7

• Did the applicant provided CNCS-requiredevaluation report that meets CNCSrequirements (if applicable)?

Yes No

• Did the applicant provided CNCS-requiredevaluation report is of satisfactory quality (ifapplicable)?

Yes No

Note: Commission of National and Community Service will provide the reviewers with the CNCS-required evaluation report requirement for each application because the it various based on how many years has program been operational. OCFS subject matter expert will also provide feedback and comment whether if the evaluation reports meet CNCS requirement and/or they are of satisfactory quality. The reviewers the final determination on score based on the information provided above.

Compliance and Accountability (0-8 allowable) Total Points Awarded: _________

Section C: Organizational Capacity (CONTINUED) 3. Culture that Values Learning (0-4) Comments:

• Did the applicant clearly describe how theapplicant’s board, management, and staffcollects and uses information for learningand decision making?

Yes No

Culture that Values Learning (0-4 allowable) Total Points Awarded: _________ 4. Member Supervision (0-4) Comments:

• Did the applicant clearly describe how theAmeriCorps members will receive sufficientguidance and support from their supervisorto provide effective service?

Yes No

8

• Did the applicant clearly describe how theAmeriCorps supervisors will be adequatelytrained/prepared to follow AmeriCorps andprogram regulations, priorities, andexpectations?

Yes No

Member Supervision (0-4 allowable) Total Points Awarded: _________

Section C Organizational Capability (0-25 points allowable for 4 subsections) TOTAL POINTS AWARD: ________

Section D: Cost Effectiveness and Budget Adequacy (1 subsection)Note: Reviewers will assess the quality of the application’s budget to the following criteria below. Do not assume all sub-criteria are of equal value. This criterion will be assessed based on the budget submitted. No narrative should be entered in the narrative box except for “See budget.”

1. Cost Effectiveness and Budget Adequacy (0-25 points) Comments:

• Is the budget submitted withoutmathematical errors and proposed costs areallowable, reasonable, and allocable to theaward?

Yes No

• Is the budget submitted with adequateinformation to assess how each line item iscalculated?

Yes No

• Is the budget submitted in compliance withthe budget instructions? Yes No

• Is the match information submitted withadequate information to support the amountwritten in the budget?

Yes No

9

• Is the budgeted match equal to or morethan the required match for the givenprogram year?

• Is the cost per MSY equal or less than themaximum cost per MSY stated in the 2021-22 NYS AmeriCorps Competitive RFP?

Yes No

• Did the applicant clearly provide the currentindirect rate cost if used to claim indirect /administrative costs?

Yes No

Section D: Cost Effectiveness and Budget Adequacy (CONTINUED)

• Did the applicant clearly Identify the non-CNCS funding and resources necessary tosupport the project, including for FixedAmount applicants?

Yes No

• Did the applicant clearly Indicate theamount of non-CNCS resourcecommitments, type of commitments (in-kindand/or cash), the sources of thesecommitments, and if the commitments areproposed or secured?

Yes No

Cost Effectiveness (0-25) Total Points Awarded: _________

Section D Cost Effectiveness and Budget Adequacy (0-25 points allowable for 1 subsection) TOTAL POINTS AWARD: ________

Yes No

DRAFT

Page 1

STATE APPLICATION IDENTIFIER:

FEDERAL IDENTIFIER:

21ES231071

2b. APPLICATION ID:

24-NOV-20

3. DATE RECEIVED BY STATE:

4. DATE RECEIVED BY FEDERAL AGENCY:

City Year, Inc.LEGAL NAME:

287 Columbus Avenue

Erin E. BennettNAME:

(617) 927-2373

TELEPHONE NUMBER:

(617) 927-2520 FAX NUMBER:

[email protected] E-MAIL ADDRESS:

2228825496. EMPLOYER IDENTIFICATION NUMBER (EIN): 7. TYPE OF APPLICANT:

8. TYPE OF APPLICATION (Check appropriate box).

If Amendment, enter appropriate letter(s) in box(es):

94.00610a. CATALOG OF FEDERAL DOMESTIC ASSISTANCE NUMBER:

AmeriCorps Fixed Amount Grant (State)10b. TITLE: City Year New York FPG

11.a. DESCRIPTIVE TITLE OF APPLICANT'S PROJECT:

New York City, NY - Schools based in Bronx, Brooklyn, Manhattan, and Queens

12. AREAS AFFECTED BY PROJECT (List Cities, Counties, States, etc):

07/01/21START DATE: 06/30/22END DATE: MA 07

$ 4,156,500.00a. FEDERAL

$ 0.00b. APPLICANT

$ 0.00c. STATE

$ 0.00d. LOCAL

$ 0.00e. OTHER

$ 4,156,500.00g. TOTAL

Jessica Greenfield

a. TYPED NAME OF AUTHORIZED REPRESENTATIVE: b. TITLE:

(617) 927-2318

c. TELEPHONE NUMBER:

11/16/20

e. DATE SIGNED:

Service/Civic OrganizationCommunity-Based Organization National Non-Profit (Multi-State)

2a. DATE SUBMITTED TO CORPORATION FOR NATIONAL AND COMMUNITY SERVICE (CNCS):

1. TYPE OF SUBMISSION:

Non-Construction

5. APPLICATION INFORMATION

NAME AND CONTACT INFORMATION FOR PROJECT DIRECTOR OR OTHER PERSON TO BE CONTACTED ON MATTERS INVOLVING THIS APPLICATION (give area codes):

ADDRESS (give street address, city, state, zip code and county):

A. AUGMENTATION B. BUDGET REVISION

C. NO COST EXTENSION D. OTHER (specify below):

9. NAME OF FEDERAL AGENCY:

Corporation for National and Community Service

13. PROPOSED PROJECT: 14. CONGRESSIONAL DISTRICT OF:

15. ESTIMATED FUNDING: 16. IS APPLICATION SUBJECT TO REVIEW BY STATE EXECUTIVE ORDER 12372 PROCESS?

YES. THIS PREAPPLICATION/APPLICATION WAS MADE AVAILABLE TO THE STATE EXECUTIVE ORDER 12372 PROCESS FOR REVIEW ON:

DATE:

17. IS THE APPLICANT DELINQUENT ON ANY FEDERAL DEBT?YES if "Yes," attach an explanation. NOX

18. TO THE BEST OF MY KNOWLEDGE AND BELIEF, ALL DATA IN THIS APPLICATION/PREAPPLICATION ARE TRUE AND CORRECT, THE DOCUMENT HAS BEEN DULY AUTHORIZED BY THE GOVERNING BODY OF THE APPLICANT AND THE APPLICANT WILL COMPLY WITH THE ATTACHED ASSURANCES IF THE ASSISTANCEIS AWARDED.

APPLICATION FOR FEDERAL ASSISTANCE

Boston MA 02116 - 5114

NEW

CONTINUATION AMENDMENT

X

7b.

7a. Non-Profit

f. PROGRAM INCOME $ 0.00

PART I - FACE SHEET

DUNS NUMBER: 622374122

Modified Standard Form 424 (Rev.02/07 to confirm to the Corporation's eGrants System) Application

NEW/PREVIOUS GRANTEE

11.b. CNCS PROGRAM INITIATIVE (IF ANY):

X NO. PROGRAM IS NOT COVERED BY E.O. 12372

d. SIGNATURE OF AUTHORIZED REPRESENTATIVE:

Year #:

X

1

a.Applicant b.Program NY 10

County: Suffolk

UEI NUMBER:

DRAFT

Page 2

For Official Use Only

Narratives

Executive Summary

Rationale and Approach/Program Design

City Year New York proposes to have 255 full-time AmeriCorps members who will provide full-day

education services to students in 20 public schools in New York City (NYC). At the end of the first

program year, as Student Success Coaches, the AmeriCorps members will mentor 2,550 students, 322

of whom will improve literacy skills and 312 will improve math skills; and, all will be supported in the

development of social-emotional skills that improve school engagement. Members will also provide

extended day and school climate activities. In addition, the AmeriCorps members will leverage 200

volunteers who will be engaged in service projects in our school communities. This program will focus

on the CNCS focus area of Education. The CNCS fixed-price investment of $4,156,500 will leverage

$4,156,500 in city funding.

1.THEORY OF CHANGE & LOGIC MODEL: RESPONSIVE TO COMMUNITY NEED: Students

who are at risk to drop out of school before graduation can be identified as early as third grade by

performance in the commonly accepted early warning indicators of academic performance in English

and math, attendance, and behavior. Within City Year New York's (CYNY) current school partners,

prior to COVID-19 closures and distance learning, 61% of 3rd to 8th grade students were not proficient

or partially proficient in English on the 2018-19 New York standardized tests compared to 55% testing

the same statewide and 64% were below proficiency in math compared to 53% statewide. COVID-19

is expected to widen these pre-existing inequalities in education and be particularly damaging to low-

income and students of color. In 2018-19, over 87% of our students were economically disadvantaged

and 97% represented minority populations. [New York City Department of Education (NYCDOE),

nycenet.edu] As an education-focused nonprofit organization, our mission is to help school districts

improve graduation rates and advance racial equity in education by bridging the gap between what

their students need and what most urban schools serving low-income populations are designed and

resourced to provide. Through our holistic model, Whole School Whole Child (WSWC), AmeriCorps

members serve in schools where they provide academic and social-emotional support to students,

serve as an essential resource in the classroom, and contribute to a positive school culture, climate,

and community. The social-emotional component of our program and our focus on creating positive

learning environments are particularly critical at this time in supporting students with overcoming

learning loss due to school closures (Durlak et al., 2011; Taylor et al., 2017). INTERVENTION:

CYNY requests to field 255 full-time members, who will be deployed in teams to their assigned school

DRAFT

Page 3


Narratives

for the entire academic year. The tiered WSWC structure establishes a preventative and intervention-

based framework through which members 1) as Student Success Coaches, are matched with students

to deliver targeted mentoring in academics, attendance, and/or social emotional development, 2)

serve in classrooms daily where they support classroom routines, positive behavior, and teachers'

instruction, 3) support before and after school programming, which includes homework assistance,

and, 4) work with school leaders to create positive school climate. Student Success Coaches provide

asset-based mentoring, including coaching on attendance, to their assigned students throughout the

day and provide 40 to 90 minutes of ELA or math support using evidence-based practices to each

mentee each week. Enrichment programming and positive climate activities are selected by the

individual school and can include: Morning Greeting, breakfast and homeroom engagement, recess

and transition time support, parent and family engagement activities (e.g. family nights, phone calls

home, and open houses), enrichment programming (e.g. talent shows, afterschool programs, field

trips, sports, student clubs, etc.), and, events that engage all students (e.g. assemblies, Poetry Month,

Spelling Bees) and/or recognize positive behaviors, such as attendance and academics awards, VIP

lunch clubs, and the award store. If needed, as a result of COVID, many of our services have been

adapted for delivery through in-person or virtual platforms. All services are selected in coordination

with the individual school so that they do not duplicate or displace existing school services or staff.

Periodic activities occur outside of the member regular schedule such as evening and weekend events

at our schools, and occasional CYNY staff approved additional service opportunities with city and

community-based organizations, such as our school Saturday tutoring programs and food banking in

our communities. CYNY has three types of member roles: 1) First Year Member, 2) Second Year

Member, and 3) Team Leader. All members support school-based activities as described above. Second

Year and Team Leaders serve as peer mentors and team coordinators and participate in advanced

level professional development and equity programs. In addition, Team Leaders provide logistical

leadership for our afterschool programs, lead team meetings and peer trainings, and represent the

team at CYNY and school meetings. VOLUNTEERS: Members will engage 200 volunteers through

physical service activities in our school communities and on national service days. All activities are

approved by CYNY staff to ensure that they do not include prohibited activities. THEORY OF

CHANGE: As the holistic WSWC framework was developed using evidence informed practices, our

theory of change was developed through external evaluation (Brett Consulting Group, 2008, 2009,

and 2010) and input from thought leaders in education. The WSWC theory of change is that when

near peers, our members, mentor students using social-emotional informed practices, students will

DRAFT

Page 4


Narratives

develop self-identity, a sense of agency, and academic and life skills, such as classroom participation,

social skills, problem solving, and critical thinking, which will make the student more likely to enter

tenth grade prepared to successfully graduate from high school and navigate college and career

(Learner, 2004, Balfanz & Byrnes, 2013). EXPECTED OUTCOMES: On an annual basis we expect to

see meaningful short-term improvement related to literacy and math and gains in attendance and

social-emotional development. Over multi-year exposure to our program, we expect to change the

mindset of students so that they are more motivated to learn and succeed academically and attend

school consistently. Our long-term goal is that by serving alongside students from elementary to high

school, students will graduate at higher rates, be better prepared, and have a more positive viewpoint

of their post-secondary success and career. RATIONALE FOR USING AMERICORPS: The full-time

aspect AmeriCorps program provides NYCDOE schools a cost-effective approach that offers wrap

around services to students and meets the district commitment of creating and supporting learning

environments that reflect the diversity of NYC. For example, we support the NYC Department of

Youth and Community Development (DYCD) in delivering the SONYC afterschool program to

middle school students. Through the AmeriCorps program, we can recruit members who are closer in

age and experiences to students than teachers and have members support school efforts to be

culturally responsive, while teaching students how to resolve conflict and build positive relationships.

CONTRIBUTIONS TO EXISTING EFFORTS: Since 2014, central to the Mayor and Chancellor's

vision for education in New York is the development of Community Schools, which provide wrap

around services to students beyond the traditional school offerings, including social-emotional and

mental health support, parent empowerment, and extended learning time. At the heart of the

Community Schools holistic model is a partnership between a community-based organization (CBO)

and school leadership. CYNY currently partners with eight Community Schools and is the lead CBO

partner at three of the eight schools. For 2021-22, we have applied to be the lead partner in up to

twenty Community Schools. Expansion of our program into more Community Schools would support

NYCDOE's efforts to reach students with critical needs that the traditional school structure and

families are not equipped to provide. PERFORMANCE MEASURE SELECTION: CYNY is submitting

two aligned performance measures that corresponds to our academic mentoring in English

(ED1A/ED5) and math (ED1A/ED5). While we provide social-emotional and attendance mentoring,

we are not providing aligned measures for these activities at this time due to our on-going learning

around objective outcome instruments in these areas. The selected measures reflect our significant

program activities and the role of the Student Success Coach, in which members support students

DRAFT

Page 5


Narratives

with their social-emotional development, academic outcomes, and school engagement on a daily

basis. RATIONALE FOR TARGETS: The proposed targets are based on prior year data, projected class

schedule, and member capacity. Our AmeriCorps members have a "Focus List" of up to thirteen

students who they mentor daily in one or more indicators, e.g. English, math, etc. The improvement

rates and definitions are realistic and ambitious in terms of the academic starting point of our students

and factors outside of our control, e.g. as a result of COVID-19 school closures and inconsistent

distance learning, low-income students may fall 12.4 months behind in catching up compared to a

projected national seven-month average for all students (McKinsey & Company, June 2020).

2. EVIDENCE BASE: Our program has strong evidence and CYNY is included in three of the four

studies which include City Year on the AmeriCorps Evidence Exchange (Policy Studies Associates,

2014; Abt Associates, 2016, MDRC, 2016, Chapin Hall, 2017). For this application, we are submitting

two evaluation reports. The findings of the two studies align with the outcomes in our logic model and

the tiered structure of our program design that includes both prevention and targeted mentoring

around academics, attendance, and social-emotional development. CYNY data is included in both

studies. STUDY #1: "Analysis of the Impacts of City Year's Whole School Whole Child Model on

Partner Schools' Performance", Policy Studies Association (PSA), 2015. STUDY DESIGN: This

evaluation is a Quasi-Experimental Design (QED) study of the City Year program in multiple

locations over multiple years. The study assessed publicly accessible testing data in English and math

over three years, 2012 to 2014. The analyses focused on approximately 600 schools in 22 cities and

included 150 schools that partnered with City Year and 460 comparison schools that did not. This

study was placed in the strong evidence tier by CNCS for the 2019-20 and 2020-21 program years.

KEY FINDINGS: This study included multiple statistically significant positive findings (e.g. p

DRAFT

Page 6


Narratives

skills and multiple measures of education outcomes (attendance, course grades, and achievement

tests). City Year provided ECG with data collected for 38,131 students (grades 3-10) who participated

in our programs. Data included student-level data on literacy and math assessments, course grades,

attendance rates, social-emotional skills observed or self-reported, student demographics, and hours of

intervention received. KEY FINDINGS: Analysis found that there was a statistically significant

relationship between students' social-emotional skills and their academic outcomes (effect size range

0.08 to 0.42), which is estimated to be the equivalent of an entire school year of academic

achievement in English or math. The study also found that the more hours students spent with City

Year members, the more likely they were to improve social-emotional levels (effect size 0.06 - 0.08;

2-3 percentile points) and English or math (effect size 0.08-0.09; 2-4 months of learning), and, the

less likely they were to be off-track in English class (odds-ratio = 0.58) and math class (odds-ratio =

0.66). These findings validate that the WSWC holistic approach has a positive impact on student

social-emotional development and academic success. SAME INTERVENTION: The beneficiary

populations in both evaluations are similar to our current school partner populations in terms of state

standardized test performance data, student enrollment, grade ranges served, and student

demographic data including race and low-income status. The member population continues to be

young adults (under 26 years old) and diverse across racial, ethnic, and education backgrounds. The

overall structure, design, and dosage of WSWC is the same. City Year conducts an annual

implementation assessment (survey completed by the supervisor of each school team) to ensure that

the model is consistent. OTHER EVIDENCE: Between 2017 and 2019, CYNY served, on average each

year, 908 students in ELA, 803 in math, and 522 in attendance, of which 36% improved in ELA, 51%

in math, and 30% in attendance. In 2020, we did not receive end of year data. CYNY was included in

a randomized control trial of the Diplomas Now partnership which indicated that students receiving

WSWC services are more likely to report positive relationships with adults (p=0.011) (MDRC, June

2016). Within our schools, a strong majority of teachers and administrative staff agreed or strongly

agreed that CYNY members help improve learning environments, improve student engagement and

participation in their academic learning, and positively impact students' overall academic

performance. In addition, schools reported that our AmeriCorps members serve as role models for

students who lack other positive role models in their lives (PSA, 2014).

3.NOTICE PRIORITY: CYNY aligns with evidence-based programs under the Education area

funding priority (see Evidence Base). Under the COVID priority area, our focus on social emotional

and relationship development will support students in re-engaging with their academic experience,

DRAFT

Page 7


Narratives

thereby helping them recover from learning losses due to COVID (Houghton Mifflin Harcourt, 2020).

4.MEMBER EXPERIENCE: SKILL DEVELOPMENT: All City Year sites share a common set of

mission-aligned beliefs and values that shape the member experience. CYNY members participate in a

six-week gradual release basic training academy hosted at Hunter College, a mid-year summit, and

learning and development workshops that foster service and professional skills, build community, and

allow members reflect on their experience. We use the GROW coaching strategy designed for young

adults to help members structure their professional aspirations and process their service experience.

Our alumni report that the most valuable skills gained were in team collaboration and leadership by

managing a group of their peers through team Coordinator Roles, group planning for afterschool

programs, participating in 360 Degree feedback protocols and Behind Closed Doors workshops, and

providing thoughtful feedback and input on our program. All members participate in our Leadership

After City Year (LACY) program, which provides workshops that advance members' professional and

personal development, e.g. resume writing, financial literacy, personal and professional brand, and

alumni and community leader panels. Through our member development programs, we are able to

engage community partners such as iMentor, Teach for America, NYC Teaching Fellows, NYC

Service, Deloitte, and NY Life. RECRUITMENT & INCLUSION: CYNY seeks applicants from

different backgrounds, abilities, education levels, geographic location, and race/ethnicity. We have

two recruiters focused on recruitment from CYNY's communities and leverage City Year's national

recruitment partnerships and our alumni board to create a diverse corps that reflects the demographic

communities we serve. Applicants receive individualized support throughout the application process

and are engaged through team shadow days, info sessions, and invitations to CYNY events such as

volunteer events and graduation. To create an inclusive community, members serve on diverse teams;

wear uniforms; utilize standardized meeting protocols that promote inclusion; participate in CYNY

and City Year network-wide affinity group discussions, equity and diversity learning activities, and

community meetings through in-person or on virtual platforms; and, have small-group facilitated

discussions that reflect on individual identity through our Idealist's Journey curriculum. Members

have access to mental health supports through our employee assistance program, Talkspace

partnership, and monthly sessions provided by New York University. Feedback from the corps is

solicited through our corps council, surveys, and roundtables with CYNY staff. Two members serve

on the City Year network-wide Dean's Council, which informs City Year on issues facing the corps.

CYNY members will be connected to other NY AmeriCorps programs through participation in the NY

Service Summit and national service day events.

DRAFT

Page 8


Narratives

Organizational Capability

1.ORGANIZATIONAL BACKGROUND AND STAFFING: City Year New York is an operating site

within the national 501c3 non-profit organization City Year, Inc. Founded in 1988, City Year is

managed by a National Board of Trustees and each operating site, such as CYNY, is led by an

Executive Director. To gain community input and support, CYNY maintains five advisory boards:

leadership, associate, investment, women's, and alumni. In 2021-22, the CYNY program staffing

model provides sound oversight and is split into the functional areas of service delivery, partnerships,

learning and development, data and evaluation, member experience, and human resources and

operations. The Managing Director of Impact (MDI) oversees thirty-eight Impact staff and is

responsible for overall program leadership and management and school district and AmeriCorps

relations. Our twenty school teams are split into two initiatives, Community Schools and SONYC

Schools, each led by a Senior Impact Director (SID). Within each initiative, an Impact Director (4)

manages four to six school relationships and service delivery and quality. Each school campus is

supervised by an Impact Manager (see Member Supervision). Four Community School Directors will

lead partnership efforts within their portfolio of schools. Five instructional staff support member

training and development and provide on-site coaching on service delivery to staff and corps. Two

staff oversee our data collection and evaluation efforts. One staff member supports members with

navigating their service experience from enrollment to exit and supports the LACY program and

transition to alumni. The Managing Director of People and Operations leads five staff in managing all

corps member and staff human resources, office operations, and equity programs. City Year

Headquarters (CYHQ) supports CYNY by providing centralized services for criminal history checks,

member enrollments/exits, finance and payroll operations, information technology, and member

recruitment. The Managing Director of Impact has over 15 years' experience leading the CYNY's

AmeriCorps program, and, over the past decade, CYNY has managed over $37M in AmeriCorps

awards, filled 3,147 member positions and 99%+ of awarded slots, retained 88% of our members, and,

submitted >97% of enrollments and exits on time. Of this, over $21M was awarded through New

Yorkers Volunteer through State Competitive funding.

2.COMPLIANCE AND ACCOUNTABILITY: Our processes are designed to meet the requirements of

an AmeriCorps program and provide for on-going monitoring and oversight of member file

documents, criminal history checks, and service activities. To prevent non-compliance, practices

within our enterprise solution, Workday, are designed to meet federal grant regulations, such as

separation of funds and time and effort certifications. We use Truescreen and Fieldprint to conduct

DRAFT

Page 9


Narratives

criminal history checks. To detect reporting errors, CYHQ staff conduct internal audits to ensure

completion of member file documents, including criminal history checks, and at least two finance

staff review AmeriCorps invoices and financial reports. Each school completes a School Partnership

Agreement, which includes the AmeriCorps prohibited activities, non-displacement, and non-

duplication requirements. To monitor service activities and our partnership relationships on a day to

day basis, we place Impact Managers in our schools to supervise the team. Impact Directors and

Instructional Coaches visit the schools bi-weekly and we have monthly partnership check-ins with

school administration. Each December and March, the Impact Manager and Impact Director for the

school complete a service partner rubric to assess the state of the partnership. Service implementation

checklists and our observation and coaching tracker assist us with identifying school needs so that we

can direct resources and support to each team. Externally, the DYCD and the NYS Department of

Health and Mental Health tests adherence to city mandated safety protocols for our afterschool

programs through periodic unannounced visits. For financial accountability, our National Board of

Trustees maintains a finance and audit committee that meets quarterly and oversees our annual

Single Audit and approves our national internal controls and financial policies. Under our National

Direct grant, CYNY has been participating in an external evaluation through MDRC and AIR that

has an approved Alternative Evaluation Approach (AEA). Per the AEA guidance, we are submitting

an interim evaluation report.

3.CULTURE THAT VALUES LEARNING: CYNY strives to create a data informed culture, which

shapes our program development, staff and corps learning and engagement, and strategic direction.

We administer engagement and satisfaction surveys to corps members, staff, and school partner staff

at least twice annually, maintain an evaluation partnership with Glass Frog Solutions, and participate

in City Year's national evaluation and learning agenda. Analysis of the surveys and our student-level

data is regularly shared and discussed with staff, corps, and our external partners through meetings,

retreats, and newsletters. As part of our partnership with the NYCDOE, we have conducted an annual

independent evaluation of our service for over a decade. Most recently Glass Frog has completed an

independent analysis our Whole School Whole Child program for school years 2018 and 2019. These

evaluations allow us to dig into our performance measure data and understand variation in service

and impact across schools, student peer groups, and types of services received. We use the evaluation

reports to refine our service model and advance our school partnerships to maximize our impact

within the schoolhouse. To build learning communities within our corps, we split them into learning

cohorts of not more than 40 corps members who share similar service experience, e.g. same grade

DRAFT

Page 10


Narratives

Budget/Cost Effectiveness

level but from different schools. Cohorts meet at least monthly for up to 90 minutes to engage in

subject-specific learning activities, e.g. third grade social-emotional strategies, that incorporates active

engagement and application exercises. The whole corps meets in-person or virtually for Community

Meetings, which are used to educate our community about the state of our service and issues of

educational equity and share life experiences through individual testimonials. In addition to our local

learning and evaluation agenda, CYNY staff and corps participate in the City Year network-wide

communities of practice, for example, weekly or monthly calls depending on role and subject and

national and regional retreats for staff and corps that focus on data, service implementation, equity

and inclusion, and/or school relationship development. Communities of practice are designed to create

learning networks across our staff and corps that foster continuous improvement.

4.MEMBER SUPERVISION: SUFFICIENT GUIDANCE & SUPPORT: Impact Manager (IM)

supervise one school campus and provide day-to-day supervision of 9 to 21 AmeriCorps members. IMs

are in schools four to five days a week where they coach our members through their professional

development by helping members set goals, evaluating their performance, and providing them

feedback on their service. IMs have monthly one-on-ones with each member of team during which

they monitor for prohibited activities and address any school, service, or personal issues the members

may have. IMs are responsible for overseeing the member calendar, verifying member service hours

and approving timesheets, and reporting any member issues to their supervisor, CYHQ, and/or the

Office of the Inspector General as appropriate. TRAINING & PREPARATION: The first six weeks of

the program year are used for training and planning for the upcoming year. Training content includes

AmeriCorps policies, our service model and goals, NYCDOE partnerships and expectations, student

progress monitoring practices, the Clover youth development model, leading with an equity mindset,

coaching skills, crucial conversations, collaborative goal setting, and City Year policies and

information systems. The Learning and Instruction team provides training in curriculum delivery and

mentoring practices. School partners provide a start of year training that includes an introduction to

school faculty, school priorities and protocols, and teacher classroom orientations. All IMs complete

professional development plans and participate in twice monthly training opportunities. IMs have

day-to-day support from their Impact Director, Instructional Coach, and Community School Director

and have peer support through monthly Impact team meetings, national or regional training events

led by CYHQ, and twice annual Impact team retreats.

See budget.

DRAFT

Page 11


Narratives

Evaluation Summary or Plan

CITY YEAR NEW YORK 2021 EXTERNAL EVALUATION PLAN

EVALUATION PLAN SUMMARY:

EVALUATOR: MDRC and American Institutes for Research

STUDY FOCUS: Impact of the Whole School Whole Child on student and school-wide social-

emotional and academic outcomes

STUDY TYPE: Randomized control trial and quasi-experimental design study

GRANT YEARS COVERED: 2018-19 and 2019-20

APPLICABLE RECOMPETING APPLICATIONS:

* City Year Jacksonville: 18ACHFL0010001 (21AC230127)

* City Year Los Angeles: 18ACHCA0010003 (21AC230042)

* City Year New York: New/Recompete (21ES231071)

* City Year Orlando: 18ACHFL0010002 (21AC230126)

* City Year San Antonio: 18ACHTX0010002 (21AC230077)

PRIOR AEA REQUEST: Requested on the basis of timing. Following AmeriCorps AEA guidance, an

Implementation Report of work conducted to date has been submitted with the application.

***********************************

EVALUATION PLAN NARRATIVE:

PROGRAM BACKGROUND: The City Year (CY) Whole School Whole Child (WSWC) service model

is a two-tiered model providing universal and targeted academic and socio-emotional mentoring as

well as attendance-monitoring services. City Year recruits and places teams of full-time AmeriCorps

members (ACMs) in schools to serve as Student Success Coaches and deliver the WSWC model. ACMs

DRAFT

Page 12


Narratives

provide universal (Tier 1) services to all students by assisting teachers in the classroom; supporting

non-instructional time outside the classroom; and organizing school events, community service

projects, and afterschool programs. ACMs also provide individualized (Tier 2) services -individualized

mentoring that develops skills to support English, math, attendance, and social-emotional

development - to students who exhibit early warning signs for not graduating on time.

THEORY OF CHANGE: The WSWC theory of change is that when near peers, our members, mentor

students using social-emotional informed practices, students will develop self-identity, a sense of

agency, and academic and life skills, such as classroom participation, social skills, problem solving,

and critical thinking, which will make the student more likely to enter tenth grade prepared to

successfully graduate from high school and navigate college and career (Lerner, 2004). Further, when

students who are most at-risk receive individualized services in a supportive school environment,

other students improve overall attendance, behavior and course performance due to decreased

distractions for students and teachers (Balfanz & Byrnes, 2013).

OUTCOMES OF INTEREST: As part of the planned evaluation City Year has developed two logic

models to align with universal (Tier 1) services and targeted (Tier 2) services. The Tier 1 logic model

outlines whole school services and connects these program components with attended outcomes. For

example, connecting whole class support in reading and mathematics classes to schoolwide

performance on state standardized assessments. The Tier 2 model utilizes the same inputs and

activities but adds individual mentoring and student progress monitoring, which are then connected to

outcomes at the individual student level. For example, the activity for attendance is individual

coaching and mentoring versus school-wide programming and the outcome is the number of students

who improve attendance versus improvement in the schoolwide attendance rate. The long-term

outcome for all activities is to increase high school graduation rates and have more students exit high

school prepared for post-secondary success. Outcomes of interest include: improved schoolwide and

individual attendance; improved schoolwide and individual course grades and standardized test scores

in reading and math, improved individual goal mastery and growth mindset, improved individual self-

management and emotional intelligence, and improved school climate, specifically, student-adult and

student-student relationships.

RESEARCH QUESTIONS: The study of the WSWC model, both Tier 1 and Tier 2 services, will

address three impact questions: (1) Do students in City Year middle schools have better academic,

behavioral, and school engagement outcomes and experience more positive school climate than

students in comparison schools?; (2) Do the effects vary by student subgroup (age, gender, race or

DRAFT

Page 13


Narratives

ethnicity)?; and (3) Do the effects increase over time, as schools implement the WSWC model for

more years? The study will address two implementation questions: (1) To what extent are services

provided with fidelity? and (2) How do variations in implementation fidelity explain variations in the

observed effects? A cost analysis will address this research question: What are the monetary costs of

implementing the WSWC model in urban middle schools?

RESEARCH & EVALUATION DESIGN: In February 2017, the Institute of Education Sciences (IES)

Education Research Grants Program awarded a grant for a multi-year efficacy study of our Whole

School Whole Child model in high need urban middle schools to the American Institutes of Research

(AIR), MDRC, and Stephanie Jones (Harvard University). The study will evaluate the routine

conditions of WSWC and test the efficacy of City Year's WSWC model and the added value of targeted

(Tier 2) services. The study will be conducted in five districts serving large urban areas: Los Angeles

Unified School District (CA), New York City Public Schools (NY), Duval County Public Schools (FL),

Orange County Public Schools (FL), and the San Antonio Independent School District (TX). The study

includes a student-level randomized controlled trial (RCT) to assess the effects of Tier 2 services on

students after 1- and 2- years' participation, a comparative interrupted time series (CITS) design to

assess the effect of the WSWC model, and an implementation analysis to examine the costs of

implementation. The RCT includes Tier 2 student level data from the evaluation years (2018-19 -

2019-20) and the CITS includes whole school data from 2007-08 to 2018-19. The RCT was originally

planned to include two cohorts of students where cohort 1 received services in 2018-19 and 2019-20

and cohort 2 was to receive services 2019-20 and 2020-21. Due to COVID-19, the investigators

eliminated the second cohort. The CITS study was originally planned to extend through 2020-21,

however, due to COVID-19, the 2019-20 and 2020-21 years have been eliminated. The RCT is being

conducted by AIR and the CITS and implementation analysis is being conducted by MDRC.

RATIONALE: In addition to the highly rigorous RCT framework for Tier 2 services, the CITS design

is more rigorous than most other quasi-experimental design (QED) studies because it combines

longitudinal data and a matched comparison group. CITS designs have been used to evaluate federal

policies like No Child Left Behind (Dee & Jacob, 2011; Wong, et al, 2015) and grade-level reform

models (Somers & Garcia, 2016). Several studies have shown that CITS can produce statistically

similar results to an RCT under certain conditions (Fretheim et al., 2013; Schneeweiss, et al 2004;

Somers et al., 2013; St. Clair, et al, 2014), which are expected to be met in this study.

STRENGTH AND LIMITATIONS: Possible biases to the comparison group's baseline trends could

occur through policy shock or instructional reform, which may affect either the City Year group or

DRAFT

Page 14


Narratives

control schools; and / or comparison schools may experience different demographic shifts in student

populations. Due to COVID-19, several aspects of our IES evaluation have been impacted. For

example, the evaluators will be excluding the second RCT cohort from the study as services were

significantly disrupted due to school closures.

SAMPLING: The sample includes students identified in early sixth grade as needing support relative to

literacy, math, attendance, and behavior; in 22 City Year schools and approximately 30 matched

comparison schools in five urban school districts (Los Angeles, New York, Jacksonville, Orlando, and

San Antonio). The CITS study will include data from school years 2007-2008 to 2018-19 representing

nearly 300,000 students. The RCT study includes approximately 3,300 students in grades 6-8 enrolled

in these schools during the evaluation years.

DATA SOURCES: MEASUREMENT TOOLS AND DATA COLLECTION: Measurement tools will

include our standard suite of student pre/post assessments (determined by school and district), report

card grades, and observational data. Student pre- (fall) and post- (spring) surveys, as well as all

relevant student outcome and intervention data, will be collected by City Year throughout the course

of the study and shared with the researchers to run analyses and report out results. Whole school data

will be collected through public resources (state department of education websites) by the researchers.

To accommodate changes in school assessment schedules and testing due to COVID-19, for 2019-20,

mid-year data may be used instead of end of year. IES is aware that study years have changed as a

result of COVID-19.

ANALYSIS PLAN: Both the RCT and CITS studies will use hierarchical linear modeling to account for

clustering of data. The CITS study will be used to evaluate the effects of the full WSWC model. Both

studies will conduct student subgroup analyses and will document implementation fidelity. Both

studies will measure student outcomes in three domains: academic achievement; social, emotional,

and behavioral; and school engagement.

EVALUATOR QUALIFICATIONS:

American Institutes for Research: AIR has earned an international reputation for advancing

educational science and has evaluated programs addressing middle school improvement, dropout

prevention, and high needs youth. AIR has conducted over six RCTs, five QEDs, two CITS, and 15

mixed-methods studies that address social-emotional development in secondary schools.

MDRC: MDRC is an education and social policy research organization with a 40+-year history of

conducting large-scale demonstrations and evaluations. MDRC is widely considered a leading

institution in rigorous evaluations of school reforms, has conducted numerous large-scale

DRAFT

Page 15


Narratives

experimental and quasi-experimental evaluations within schools, and has worked with hundreds of

middle and secondary schools.

The seven team members included in this proposal have expertise in the areas necessary for making

this project a success. David Osher, Ph.D., Vice President at AIR, will serve as the Principal

Investigator (PI) with Marie-Andrée Somers, Ph.D., Senior Research Associate at MDRC, as a Co-PI.

TIMELINE AND BUDGET: The IES award period is 9/1/2017-12/31/2021. The planning year was

held in 2017-18. The evaluation years span 2018-19 to 2020-2021. The report will be written during

the 2021-2022 program year. Reports are scheduled: interim report, Year 3, final report, Year 4, and a

manuscript for peer review, Year 5. The IES grant award budget is $3.2 million. Timeline, report

schedule, and budget subject to change per MDRC and AIR negotiation with IES.

*****************************************

REQUEST FOR ALTERNATIVE EVALUATION APPROACH:

City Year New York has operated through City Year's National Direct grant over the previous grant

cycle. Under our 2018 National Direct Recompete Application, we submitted an Alternative

Evaluation Approach (AEA) on the basis of timing for the IES evaluation as it is a five-year study. As

this study will continue into the next grant cycle, we are requesting to renew the AEA on the basis of

timing. In addition, City Year New York has already completed a prior experimental impact

evaluation that demonstrates evidence of effectiveness in one or more key desired outcomes that are

aligned with our theory of change. Evaluation constraints, rigor, and contribution to City Year's

learning base are as described:

A) CONSTRAINTS: The constraints relate to funding, capacity, and redundancy. City Year invests in

national studies to conduct experimental and rigorous evaluations of the program design as a whole

and localized studies with lower rigor to test specific components of the program design. Over the past

three years, City Year, Inc. has invested in six multi-site external evaluations through the Everyone

Graduates Center, Policy Studies Associates, Research Triangle Institute, Deloitte Consulting, and

MDRC and AIR. Of these, one has been completed and the remaining five are scheduled to be fully

completed by the summer of 2022. Our organizational staffing and financial capacity to add another

evaluation at this time are limited. For the proposed IES study, the lead investigators, MDRC and

AIR, established the five-year time horizon and set the schedule for reports. In addition, following

DRAFT

Page 16


Narratives

standard practices during randomized control trials, MDRC and AIR will have locked data sets

meaning that we cannot share the data for other research evaluations during the five-year study

period, which limits both our ability to receive reports and to conduct other evaluations using the

same data.

B) RIGOR: The national evaluation approach provides us the opportunity to secure grants through

the U.S. Department of Education that support high quality and rigorous evaluations, such as the

multi-year randomized control trial described in the evaluation plan, that we otherwise would not

have the resources or capacity to implement. A randomized control trial is the highest level of rigor.

By comparing the characteristics of each evaluation study's cohort of City Year schools and districts

with those served by City Year but not participating in the study, we infer that the conditions our

members operate in will be sufficiently comparable regardless of direct study involvement. We can

also reasonably conclude that program impact will be consistent regardless of location. This

conclusion has been supported by evaluator MDRC's Principal Investigator in reference to our

participation in the national evaluation of Diplomas Now (i3).

C) CONTRIBUTION TO EVIDENCE BASE: All City Year evaluations, local or national, contribute to

the organizational body of evidence and inform on our learning agenda. City Year leverages research

to inform practice and engages in evaluation to better understand our impact and continuously

improve the services we provide to AmeriCorps members and students. Each study, including the

proposed study, is used to inform on our learning agenda and our next study's research questions and

test assumptions. For example, future areas of inquiry include the role of learning environments and

relationships in shaping student, educator, and AmeriCorps member experiences, the role of

communities in supporting school outcomes, understanding effective integrated social, emotional, and

academic strategies, and how service impacts member future experiences. The IES fills a gap in our

evidence in that City Year does not currently have experimental research evidence on the effectiveness

of WSWC as a whole. The IES efficacy study evaluates the effects of the WSWC model and the added

value of Tier 2 services. The proposed project is important to address the need, population, evaluation

design, timing of outcomes, and the social and behavioral context for academic learning.

TIMING AEA:

As included in our Evaluation Plan, the IES study is anticipated to take 5 years with a report being

issued in 2021 and a peer-reviewed manuscript to be issued in 2022. The final report will be available

DRAFT

Page 17


Narratives

at our next recompete application in fall 2023. As noted, we do not control the timeline for this

evaluation and have limitations on using data for the study operating sites until the evaluation is

complete.

PRIOR EVALUTION AEA:

City Year has a robust evaluation and research program that ensures that all sites are included in

studies that are rigorous and contribute to our evidence base. City Year New York has participated in

a prior randomized control trial and two quasi-experimental design studies as well as non-

experimental studies. A selection of studies includes:

EVERYONE GRADUATES CENTER: "Connecting Social-Emotional Development, Academic

Achievement, and On-Track Outcomes: A Multi-District Study of Grades 3 to 9 Students Supported by

City Year", 2020.

* Study Type: Non-Experimental Mixed Methods

* Evidence of Effectiveness: This study demonstrated that for students who participate in our program

there are statistically significant and consistent relationships between students' social-emotional skills

and their academic outcomes across grade levels, and that, the more hours of support a student

receives from an AmeriCorps member, the higher the student outcomes in the related subject (ELA or

math), attendance, and social-emotional skills.

GLASS FROG SOLUTIONS, "Examining the Factors Associated with Student Growth at City Year

New York", 2019

* Study Type: Non-Experimental Quantitative Analysis

* Evidence of Effectiveness: This study was designed to identify best practices to inform decision-

making within each of our programmatic areas. The findings show that CYNY has year over year

consistency in outcomes across indicator areas and that certain subgroups of students and assessment

tools have consistently higher outcomes than others. The study included data from program years

2015-16 to 2018-19.

MDRC, "Addressing Early Warning Indicators: Interim Impact Findings from the Investing in

Innovation (i3) Evaluation of Diplomas Now", 2016.

DRAFT

Page 18


Narratives

Amendment Justification

Clarification Summary

Continuation Changes

* Study Type: Randomized Control Trial

* Evidence of Effectiveness: This study demonstrated that our program had a statistically significant

impact on reducing the percentage of sixth and ninth graders demonstrating early warning indicators

and on reducing chronic absenteeism.

* Evidence Tier: Under the 2018 grant competition, this study was assessed by CNCS to be in the

Strong evidence tier and is posted on the Evidence Exchange.

POLICY STUDY ASSOCIATES, "Analysis of the Impacts of City Year's Whole School Whole Child

Model on Partner Schools' Performance", 2015.

* Study Type: Quasi-Experimental Design

* Evidence of Effectiveness: This evaluation demonstrated that our program had statistically

significant impact on improving whole school performance on literacy and math assessments.

* Evidence Tier: Under the 2019 and 2020 grant competitions, this study was assessed by CNCS at the

Strong evidence tier.

POLICY STUDIES ASSOCIATES, "Year 4 Evaluation of City Year New York's Whole School Whole

Child Initiative", 2014

* Study Type: Non-Experiment and Quasi-Experimental Design

* Evidence of Effectiveness: This study focused on implementation and identifying areas for

improvement. In addition, it provided additional analysis of City Year performance measure data and

City Year administered surveys. The analysis found that schools believe that City Year has an impact

on student academic outcomes and academic engagement.

* Evidence Tier: This evaluation was assessed at the preliminary evidence tier and is posted on the

Evidence Exchange.

Not applicable.

Not applicable.

Not applicable.

DRAFT

Page 19


Grant Characteristics

DRAFT

Page 20


Education

Focus Area

100%

% MSYsK-12 Success 100%

K-12 Success

Objectives

255.00

No of MSYs

255

No of Members

Table1: MSYs by Focus Areas Table2: MSYs by Objectives

Table4: No of MSY and Members by Objective

NPM Applicant 69% 0%% MSYs

Table3: %MSYs by NPM vs.Applicant vs. Not in ANYNot in ANY 31%

Primary Focus Area:

Secondary Focus Area:

Education

Education

Primary Intervention:

Secondary Intervention:

Mentoring

Other Classroom Support

Performance Measures

Objectives %MSYs

255.00Total 255

DRAFT

Page 21


Problem Statement:

Selected Interventions:

MentoringOther Classroom Support

Academic Services in English Language ArtsPerformance Measure:

Education K-12 SuccessFocus Area:

Objective:No of Members:

93.00 148No of MSY's:

Describe Interventions:

DESIGN: City Year New York members are embedded in 3rd to 10th grade classrooms for English Language Arts where they serve as Student Success Coaches and provide focused academic mentoring in literacy and whole class support. In classrooms, members support the whole class in following instructional routines and provide small-group focused support using NYCDOE literacy protocols. Mentoring includes improving durable literacy skills and supports development of good study habits, such as note taking, collaboration, task prioritization, and homework completion. Member activities improve student on-task behavior and limit peer distraction while addressing student motivation and academic engagement, such as in-class assignment completion and participation. Members also model good academic behaviors and reinforce the concepts of a collaborative classroom climate. Outside of the classroom, members develop a culture of literacy through literacy-based events and enrichment programming. DOSAGE: All students in the classroom are substantially engaged through full-time placement of members in the classroom for the academic year. Classes are 45-90 minutes per class five days per week. Focused literacymentoring occurs in 30 to 45-minute sessions 2x per week. Each student included under ED1A is expected toreceive at least 60 minutes of individualized support per week. Members may also provide literacy support to

In 2021-22, City Year New York anticipates partnering with 20 elementary, middle and K-8/6-12 schools (5 are TBD) within the neighborhoods of Queens, Harlem, Brooklyn and the Bronx, in order to improve student outcomes that lead to improved graduation rates. In 2018-19, City Year’s proposed partner schools enrolled nearly 7,000 students, the majority of whom represent minority (avg. 95%) and economically disadvantaged (avg.87.4%) backgrounds (2018-19 School Performance Dashboard, NYC DOE). Annual state testing shows that low-income and minority students perform lower on standardized testing for literacy than their more affluent and white peers and are at risk for not graduating high school on time. For example, 16% of students who are not reading at proficiency by the end of 3rd grade will not graduate high school on time, however, among low-income students in general, 26% will not graduate on time and among low-income students who are also livingin neighborhoods of concentrated poverty, 35% will not graduate on time (Hernandez, 2012). On the 2018-19 New York State Testing Program (NYSTP), 61% (3,008/4,949) of students tested in 3rd-8th grades at City Yearpartner schools scored below proficiency (Partially/Not Proficient) on the English portion of the exam. In comparison, 55% (539,119/987,398) of 3rd-8th grade students statewide, and of these, 53% (206,434/392,460) of students citywide tested below proficiency in ELA this same year (2018-19 School Performance Dashboard, NYC DOE). The city’s overall graduation rate has increased to a high of 77.3% in 2019, but is still significantly lower than the state’s graduation rate of 83.4% (2019 NY State Graduation Rate Data). The Early Warning Indicators predict that students who are below proficiency or failing English are at high risk to drop out (Balfanz & Neild, 2006). Further, early intervention in English before the 9th grade is key to getting struggling students back on track for graduation (Corrin, et al, 2016).

DRAFT

Page 22


Problem Statement:

Selected Interventions:MentoringOther Classroom Support


Described Instrument: TITLE: CYSCHOOLHOUSE. cyschoolhouse is a secure cloud-based platform thatprovides an intervention management and tracking system to record student-level data.WHAT IS MEASURED: Student profile data includes name/identifying number, school, grade level, member assignment, which targeted interventions students receive, and participation. The Gradebook module of cyschoolhouse tracks individual student grades and an Assessment module tracks attendance data, other school based data, and assessments.HOW ADMINISTERED: cyschoolhouse is managed by City Year Headquarters. City Year New York staff enter student profile and school data with records provided by schools/district. Intervention data is entered by members using computer, tablet, or mobile phone application. RELIABILITY/VALIDITY: Only one record per student is created to ensure non-duplication, further City Year Headquarters cleans data to make sure that recordsare valid and not duplicated. Data is reviewed and analyzed by both City Year NewYork and Headquarters analytics staff. cyschoolhouse is FERPA compliant and was designed by Acumen Solutions for City Year in 2012 and vetted by a consortium of DC Charter schools. TARGET: Approximately 125 members will be assigned to ELA classrooms where they support all students in the classroom (25-30 students per member) and are assigned up to 10 Focus List students for additional intensive literacy supportabove standard strategies, e.g. outside of class. Targets are based on the subset of students assigned to members in English classrooms and reflect variations in school enrollment size, class schedule, and member capacity. Team Leaders do

ED1A: Number of individuals served

ED1A Output:

Tracking SystemMeasured By:

1072Target: Individuals

students outside of class time, e.g. before or after school. Once enrolled in our program, students are expectedto receive literacy services for the duration of the school year unless they withdraw from school or are assignedto another classroom. Combined class and focus time, students receive a minimum of 5 hours of literacy services per week.NYCDOE PROTOCOLS: Members will support students through use of the Levelled Literacy Intervention System and/or City Year’s research-based literacy protocols in fluency, vocabulary, and reading comprehension.

DRAFT

Page 23


Problem Statement:



Described Instrument:


not carry assigned Focus List students but engage all Focus List students in the school.

TITLES: Outcome instruments are selected in partnership between City Year NewYork, the New York City Department of Education, and individual schools. Instruments: Fountas & Pinnell Reading Record (F&P) or English course grades.WHAT IS MEASURED: All instruments measure student level progress in English. Assessments measure skills and knowledge in the domains of literacy as applicable to the grade level: phonemic awareness, phonics, fluency, vocabulary, comprehension, and writing. Grades measure content mastery, homework completion, and participation. Improvement is defined as increasing by 1 reading level (F&P), or improving course grade or grade point average (depending on alphaor point grading system).HOW ADMINISTERED: F&P is a computer adaptive test administered by the school up to three times a year. Grades are issued by teachers at least four times a year. The first assessment or marking period grade or, when available, prior year end of year results, will serve as baseline/pre data and the last assessment/gradefor student for the year as post data.RELIABILITY/VALIDITY: F&P has been extensively field tested with a gradient of text difficulty basis that has been developed over 20 years. Course grades are indicators or predictors of high school completion and student success in post-secondary academics (American Institute for Research, 2013). Each student will only be reported once, either assessment or grade. TARGET: The number of students improved is ambitious for student starting point, student and school conditions, and data availability. City Year is currently working with the Center for Assessment to further understand the impact of learning loss due to COVID-19 and how this will impact our expectations and definitions of improvement.

ED5A: Number of students with improved academic performance

ED5A Outcome:

Report Card Grade, Standardized TestMeasured By:

322Target: Students

DRAFT

Page 24


Problem Statement:

Selected Interventions:

MentoringOther Classroom Support

Academic Services in MathPerformance Measure:

Education K-12 SuccessFocus Area:

Objective:No of Members:

82.00 130No of MSY's:

Describe Interventions:

DESIGN: City Year New York members are embedded in 3rd to 10th grade math classrooms where they serve as Student Success Coaches and provide focused academic mentoring in math and whole class support. In classrooms, members support the whole class in following instructional routines and provide small-group focused support using NYCDOE math protocols. Mentoring includes improving durable math skills and supports development of good study habits, such as note taking, collaboration, task prioritization, and homework completion. Member activities improve student on-task behavior and limit peer distraction while addressing student motivation and academic engagement, such as in-class assignment completion and participation. Members also model good academic behaviors and reinforce the concepts of a collaborative classroom climate. Outside of the classroom, members develop a culture of math through math-based events and enrichment programming, e.g. Pi Day, etc.DOSAGE: All students in the classroom are substantially engaged through full-time placement of members in the classroom for the entire academic year. Classes are 45-60 minutes per class five days a week. Focused math mentoring occurs in 20 to 60-minute sessions 2x per week. Each student included under ED1A is expected to receive at least 40 minutes of individualized support per week. Members may also provide math support to students before or after school. Once enrolled in our program, students are expected to receive

In 2021-22, City Year New York anticipates partnering with 20 elementary, middle and K-8/6-12 schools (of which, 5 are still being determined) within the neighborhoods of Queens, Harlem, Brooklyn and the Bronx, in order to improve student outcomes that lead to improved graduation rates. In 2018-19, City Year’s proposed partner schools enrolled nearly 7,000 students, the majority of whom represent minority (avg. 95%) and economically disadvantaged (avg. 87.4%) backgrounds (2018-19 School Performance Dashboard, NYC DOE). Annual state testing shows that low-income and minority students perform lower on state standardized testing for math than their more affluent and white peers. As a result, these students are at risk for not graduating high school on time and are in need of additional academic support. On the 2018-19 New York State Testing Program (NYSTP), 64% (3,166/4,963) of students tested in 3rd-8th grades at City Year partner schools scored below proficiency (Partially/Not Proficient) on the math portion of the exam. In comparison, 53%(505,607/948,606) of 3rd-8th grade students statewide, and of these, 54% (208,480/383,235) of students citywide tested below proficiency in math this same year (2018-19 School Performance Dashboard, NYC DOE).The city’s overall graduation rate has steadily increased since 2005 to a high of 77.3% in 2019, but is still significantly lower than the state’s graduation rate of 83.4% (2019 NY State Graduation Rate Data). The Early Warning Indicators predict that students who are below proficiency or failing math are at high risk to drop out (Balfanz & Neild, 2006). Further, early intervention in math before the 9th grade is key to getting a student who is demonstrating one or more of the four key performance indicator areas (English, math, attendance, and behavior) back on track for graduation (Corrin, et al, 2016).

DRAFT

Page 25


Problem Statement:



Described Instrument: TITLE: CYSCHOOLHOUSE. cyschoolhouse is a secure cloud-based platform thatprovides an intervention management and tracking system to record student-level data.WHAT IS MEASURED: Student profile data includes name/identifying number, school, grade level, member assignment, which targeted interventions students receive, and participation. The Gradebook module of cyschoolhouse tracks individual student grades and an Assessment module tracks attendance data, other school-based data, and assessments.HOW ADMINISTERED: cyschoolhouse is managed by City Year Headquarters. City Year New York staff enter student profile and school data with records provided by schools/district. Intervention data is entered by members using computer, tablet, or mobile phone application.RELIABILITY/VALIDITY: Only one record per student is created to ensure non-duplication, further City Year Headquarters cleans data to make sure that recordsare valid and not duplicated. Data is analyzed by both City Year New York and Headquarters analytics staff. cyschoolhouse is FERPA compliant and was designed by Acumen Solutions for City Year in 2012 and vetted by a consortium ofDC Charter schools. TARGET: Approximately 107 members will be assigned to math classrooms, wherethey support all students in the classroom (25-30 students per member) and are assigned up to 8 Focus List students for additional intensive math support above standard strategies, e.g. outside of class. Targets are based on the subset of students assigned to members in math classrooms and reflect variations in school enrollment size, class schedule, and member capacity. Team Leaders do not carry assigned Focus List students but engage all Focus List students in the

ED1A: Number of individuals served

ED1A Output:

Tracking SystemMeasured By:

890Target: Individuals

math services for the duration of the school year unless they withdraw from school or are assigned to another classroom. Combined class and focus time, students receive a minimum of 4.5 hours math services in math perweek.NYCDOE PROTOCOLS: Protocols follow the five strands of mathematical proficiency: Conceptual Understanding, Procedural Fluency, Strategic Competence, Adaptive Reasoning, and Productive Disposition, derived from the National Research Council and the IES Practice Guide.

DRAFT

Page 26


Problem Statement:





school.

TITLES: Outcome instruments are selected in partnership between City Year NewYork, the New York City Department of Education, and individual schools. Instruments: HMH Math Inventory (MI) or math course grades.WHAT IS MEASURED: All instruments measure student level progress in Math. Math Inventory measures skills and knowledge in math strands as applicable to the grade level such as, counting and cardinality, fractions, operations, ratios, proportional relationships, and algebraic thinking. Grades measure content mastery, homework completion, and participation. Improvement is defined as meeting or exceeding typical growth for the grade level and starting point of the student as established by the assessment vendor (MI) or improving course gradeor course grade point average (depending on alpha or point grading system).HOW ADMINISTERED: MI is a computer adaptive test administered by the schoolthree times a year. Grades are issued by teachers at least four times a year. The first assessment or marking period grade or, when available, prior year end of yearresults, will serve as baseline/pre data and the last assessment or grade for the student for the year will serve as post data.RELIABILITY/VALIDITY: MI is a product of Houghton Mifflin Harcourt, is nationally normed, and uses the research-based MetaMetrics quantile scoring system. Course grades are indicators or predictors of high school completion and student success in post-secondary academics (American Institute for Research,2013). Each student will only be reported once, either assessment or grade. TARGET: The number of students improved is ambitious for student starting point, student and school conditions, and data availability. City Year is currently working with the Center for Assessment to further understand the impact of learning loss due to COVID-19 and how this will impact our expectations and definitions of improvement.

ED5A: Number of students with improved academic performance

ED5A Outcome:

Report Card Grade, Standardized TestMeasured By:

312Target: Students

DRAFT

Page 27


AmeriCorps Funding Priorities

Grant Characteristics

*Check any priority area(s) that apply to the proposed program. In order to receive priority consideration, applicants must demonstrate that the priority area is a significant part of the program focus, high quality program design, and outcomes.

*Check any characteristics that are a significant part of the proposed program.

Program Information

COVID-19 Recovery, Evidence-Based Intervention

Geographic Focus-Urban

DRAFT

Page 28


Other Revenue Funds

Number of episodic volunteers generated by AmeriCorps members

Ed Priority: Enter row number (1-13) of intervention in Education Evidence Brief (enter 0 for N/A)

EO Priority: Enter row number (1-4) of intervention in Econ Opp Evidence Brief (enter 0 for N/A)

HF Priority: Enter row number (1-5) of intervention in Healthy F. Evidence Brief (enter 0 for N/A)

4156500

200

0

0

0

Demographics

DRAFT

Page 29


Required Documents

Evaluation

Labor Union Concurrence

Other Documents

Evidence Report - PSA Study

Evidence Report - Everyone Graduates

Document Name

Sent

Not Applicable

Not Applicable

Sent

Sent

Status

Logic Model

The community problem that the program activities (interventions) are designed to address.

STUDENT SUCCESS COACHES: Academic performance in English and math are leading indicators of student drop out risk and students who are academically behind in middle

Resources that are necessary to deliverthe program activities (interventions), including the number of locations/sites and number/type of AmeriCorps members.

255 full-time AmeriCorps members, placed inschools (34-36 weeks), Mon-Fri.; 2 days/month are for training.

45 City Year staff positions

20 public schools receiving teams of

The core activities that define the intervention or program model thatmembers will implement or deliver, including duration, dosage and target population.

All Focus List students receive support services for the full school year:

A) ELA/Math support: academic mentoring 2x/week,20-60 min/session; extra support in daily class periods;

Direct products from program activities.

2,550 Focus List students in 3 to 10th grade, of which:

1,072 receive English Language Arts focused support and mentoring (ELA ED1A)

890 receive Math

Changes in knowledge, skills, attitudes and opinions. These outcomes, if applicable to the program design, willalmost always be measurable during the grant year.

2,550 Focus List students are routinely monitoredfor emerging trendsand improvements through our Student Progress Monitoring protocols, of which:

322 students will improve knowledge

Changes in behavior or action. Depending on program design, these outcomes may or may not be measurable during the grant year.

Students will improve behaviors that demonstrate commitment to learning and academic engagement

Students will improve behaviors in attendance

Students improve

Changes in condition or status in life. Depending on program design, these outcomes may or may not be measurable during the grant year. Some programs, such as environmental or capacity-building programs, may measure changes in condition over a period as short as one year.

80% of students who receive targeted services through middle school will not require targeted services in high school

Increased rates of on-time high school graduation

Problem Inputs Activities Outputs Short-Term Outcomes

Mid-Term Outcomes

Long-Term Outcomes


Logic Model

school are less likely to succeed in high school. (Balfanz, Herzon, & Mac Iver, 2007; Kieffer & Marinell, 2012; Holcomb-McCoy, 2007; Levin et al, 2012). Students in City Year schools are performing below proficiency on stateand district readingand math assessments, and may demonstrate poor attendance or in-school behaviors. Students need supports to help improve individual academic skills, attendance, and social-emotional development.

members plus City Year staff supervisor (Impact Manager)

Member Skills and Professional Development /Observation and Coaching Program

Research-based Literacy and Math Protocols

Student Progress Monitoring System

Clover Youth Development Framework

Check In Check Out (CICO) coaching methodology

extended learning time and homeworkhelp.

B) Student Progress Monitoring; conducted bi-weekly with teachers, 1x/monthteam review.

C) Mentoring and goal-setting programs to improve behavior and social emotional development (daily).

D) Attendance monitoring and support programs (CICO; 20 min/week).

Target Population: At risk students in grades 3 to 10.

focused support and mentoring (Math ED1A)

597 receive attendance mentoring

2041 receive social emotional mentoring

and skills in Englishbased on assessments or course grades (ELAED5A)

312 students will improve knowledge and skills in Math based on assessments or course grades (Math ED5A)

149 students will improve their presence in school based on school attendance records

behaviors in self- management, interpersonal behavior and growth mindset

Increased rates of college enrollment

Problem Inputs Activities Outputs Short-Term Outcomes

Mid-Term Outcomes

Long-Term Outcomes


Logic Model

WHOLE SCHOOL: Developmental research shows that meaningful relationships and knowledge-building foster positive feelings of connection and belonging for students (Blum & Libbey, 2004) and that supportive school climates further increase connectedness to school (Kidron & Osher, 2010). However, research also shows that under-resourced schools have a reduced capacity tocreate a climate that is conducive for stude