new york city department of education (pdf) · 2015-10-07 · nyc doe arts achieve--proposal...

NYC DOE Arts Achieve--Proposal Narrative 1

Arts Achieve: Impacting Student Success in the Arts

An Arts in Education Model Development and Dissemination Grant

New York City Department of Education

Proposal Narrative

(1) NEED FOR PROJECT

(a) The extent to which the proposed project will provide services or otherwise address

the needs of students at risk of educational failure.

The proposed Arts in Education Model Development and Dissemination (AEMDD)

project, Arts Achieve: Impacting Student Success in the Arts, will include the development of

5th

grade benchmark arts assessments and the use of data from these to ensure that students have

access to a rigorous arts program that is aligned with National, State, and local arts standards.

Arts Achieve will use a rigorous experimental design in which stratified random assignment will

be used to select and assign eligible schools to treatment or control conditions. This study will

assess whether the implementation of balanced (formative and summative) benchmark arts

assessments -- when accompanied by targeted and intensive professional development for school

staff and rigorous, sequential instruction for students -- leads to increased student achievement in

the arts, improved student outcomes in ELA and math, and enhanced essential 21st Century

Skills.

Schools will be eligible to participate in the AEMDD project if they meet the following

criteria: serve grades 3-5; have a certified arts teacher or an arts assigned cluster teacher; work

with at least one cultural partner; and offer at least 50 hours per year of one art form to students

in all grades 3-5. According to the results of analyses at the time of this writing, 247 NYC

public schools meet the eligibility criteria. By art form, 196 schools met the criteria in visual arts,

126 in music, 30 in dance, and 24 in theater.


Across the eligible schools, 87% are Title 1 eligible and 6% are designated as ―Schools in

Need of Improvement‖ based on their 2008-2009 state exam performance. A full list of the 247

eligible schools is provided in the Attachments.

The NYC DOE Office of Arts and Special Projects (OASP) will develop 5th

grade

benchmark arts assessments during the planning year of this grant to be administered in the eight

treatment schools in the spring of each project year. Administrators, arts teachers, and grade-

level teacher leaders in the treatment schools will form professional learning communities

(PLCs) both within and across schools to examine arts data from these assessments. They will

use the data to identify areas of weakness in their arts programming and to revise their art

offerings for grades 3-5 to ensure that students are receiving sufficient quantity and quality of

arts instruction to meet National, State, and local arts standards. This work will simultaneously

build students’ skills for success in the 21st Century, as well as competencies in English language

arts (ELA) and math.

This project initially will impact a total of 3,600 students, 150 teachers, and 16 school-

based administrators in NYC public schools by the end of the four-year funding period.

Furthermore, the benchmark arts assessments that will be developed and implemented as a result

of this project will have city-wide implications for the 603 elementary schools and over 60,000

5th

grade students who will participate in the assessment when it goes to scale.

Without the sequential arts instruction in public schools that this project would engender

and support, we are seriously limiting the avenues to success and constricting an education that

should encompass higher level thinking, innovation, collaboration, imagination, student

empowerment and discipline—attributes that are valued by business, higher education and

cultural communities and that are accessed through a rigorous arts education.


(b) The extent to which specific gaps or weaknesses in services, infrastructure, or

opportunities have been identified and will be addressed by the proposed project,

including the nature and magnitude of those gaps or weaknesses.

As a nation, there has been a sea change in recent decades in our perceptions of the

purpose of education. Whereas education was once used primarily to separate out the achievers

from the non-achievers, its primary purpose is now seen much differently. Changes in our

society spurred this much needed change in perceptions and, as a result, schools are now

expected to ensure that all students are successful and, accordingly, all students must meet

specified, rigorous standards (Stiggins, 2006). Over time, these changes have led to a greater

emphasis on accountability for schools and districts to produce student results. The focus on

accountability was magnified and cemented by the No Child Left Behind Act (NCLB) of 2001,

which heralded in an ―age of accountability‖ for our educational system (Case, 2005). NCLB

requires strong accountability systems to close the achievement gap and ensure that all students

are meeting increasingly rigorous academic standards. The use of data in decision making

processes is a hallmark of the Act and is, in fact, described by the US Department of Education

(USDOE) as a National Priority (Stites, Bland, & Campbell, 2009).

While the movement toward accountability and data driven decision making in education

is undeniable, the fact that there is dearth of high quality assessments in the arts has led them to

be marginalized within the school day. Regardless of the plethora of research that indicates the

importance of the arts in children’s cognitive development and the stronger and better articulated

arts standards that exist, without authentic, high quality and useful assessments to assess student

progress toward the standards, arts education will remain marginalized and never get adequate

support or play an integral role in education across the country as do those content areas that


figure into NCLB accountability practices. At this time, the NAEP Arts Assessments offer the

best indication of the state of the arts in this country. However, these assessments are limited in

their value, as they are only implemented every 10 years; furthermore, because of budget

constraints, only music and visual arts were assessed in 2008 and only with 8th

grade students in

260 schools around the country, a far cry from indicating national needs.

While there is still a long way to go, arts educators have begun to make much needed

progress toward ensuring school accountability for the arts. For example, Washington State has

developed Arts Classroom-Based Performance Assessments (CBPAs), which are aligned with

their state learning standards, to measure students’ knowledge and skills across all grades,

Kindergarten through twelfth, in visual arts, dance, music, and theater. Additionally, the South

Carolina Department of Education developed web-based arts assessments in each of the four arts

disciplines that are aligned with the South Carolina Visual and Performing Arts Curriculum

Standards. The assessments are administered to fourth-grade students and include a multiple-

choice section and two performance tasks.

New York State also has taken a very important step in the area of accountability in the

arts by identifying a set of instructional requirements, which outline the arts experiences that

schools are expected to provide to students from grades Pre-K through 12. These requirements

not only specify the skills and abilities that students should learn at each grade level, they also

specify the amount of time that should be devoted to the arts in grades K-5 and graduation

requirements for middle and high school students. While these requirements have been

extremely useful in setting expectations for the quantity of arts in which students should

participate, there are no assessments in the arts at the state level, so expectations for the quality

of arts are far less clear.


NYC has long been a leader in setting high standards and holding schools accountable for

results. In 2003, a school reform model initiated by Mayor Michael Bloomberg and Chancellor

Joel Klein was implemented to restructure and improve the NYC public schools. The

cornerstones of the plan include rigorous use of standards-based curriculum, setting the highest

standards of teaching and coaching, and instilling each participant—principals, teachers and

learners—with accountability. NYC’s emphasis on high standards is not limited to core

academic content. Educational and government leaders recognize the value that the arts have on

the skills that children need to succeed in the 21st Century. In fact, Mayor Bloomberg recently

expressed the following: ―Reading and writing are essential tools, but so is the ability to think

critically, to understand abstract concepts, to create, to innovate. These are skills that our

students need to compete in a 21st Century economy… And these are exactly the kinds of skills

that a strong arts education will develop.” (Mayor Michael Bloomberg, July 23, 2007)

For NYC, an important step toward bringing rigorous standards and accountability to arts

education was the development of the Blueprints for Teaching and Learning in the Arts. The

development of the Blueprints, which were published between 2003 and 2005, was spearheaded

by the NYC DOE’s OASP. The Blueprints set clear standards for what students should know,

understand and be able to do in each of the four art forms as they move through the school

system from Pre-K through 12th

grade. The Blueprints, which are based on National arts

standards and support the New York State Standards for Arts Instruction, identify the scope and

sequence of learning through five strands: Art Making, Literacy in the Arts, Making

Connections, Community and Cultural Resources, and Careers and Lifelong Learning.

Benchmarks for learning are delineated at four levels – 2nd

grade, 5th

grade, 8th

grade, and 12th

grade.


As a continuation of progress toward greater accountability in the arts, in 2007, NYC

implemented ArtsCount, a data collection and accountability system that supports the standards

and the Blueprint. ArtsCount incorporates arts metrics into the NYC DOE’s measurement of

school performance, establishing first-ever accountability for arts programming and signaling the

importance of the arts to a student’s overall education. Accountability is achieved through three

primary measures: School Quality Reviews, School Progress Reports, and Principal

Performance Reviews. In addition, each school has an individual arts report posted on the

statistics page of the school’s website. This report includes information on arts courses and

offered sequences, numbers of certified arts teachers, instructional hours in the arts for students,

professional development and cultural arts providers. Furthermore there is a city-wide aggregate

report, the Annual Arts in Schools report, which tracks trends and provides an analysis of arts

education across the City’s 1,600 schools. The innovative dual strategy of NYC DOE’s OASP

that emphasizes curriculum and instruction as well as accountability in the arts for schools forms

the basis for a rigorous, sequential course of study in grades preK-12 that, if delivered

effectively, would provide all students in NYC public schools with a diverse and enriching

foundation in the arts.

Perhaps not surprisingly, however, the most recent Annual Arts in Schools Report, which

was released in October 2009, revealed a gap between arts requirements and school delivery

systems. In the 2008-2009 school year, only 16% of NYC elementary schools provided arts

instruction in all four arts disciplines and met the required hours of arts instruction across the

disciplines. While, on average, the NYC schools are barely meeting the minimum instructional

hour requirements for visual arts and music, the hours are far from being met in dance and

theater. It is also important to note that only approximately three-quarters of elementary schools


have at least one full-time certified arts teacher in any art form, and analysis of the data by

discipline reveals significant gaps: 55% of elementary schools have at least one full-time

certified visual arts teacher; 45% have at least one full-time certified music teacher; 8% have at

least one full-time certified dance teacher; and 6% have at least one full-time certified theater

teacher. Furthermore, the lack of student level arts achievement data at the school level creates

obstacles for schools in ascertaining the quality of the existing arts instruction and in the school

leader’s ability to devise a plan to address instructional gaps in the arts.

The goal of Arts Achieve is to increase accountability in the arts and, in doing so,

increase the quantity and quality of arts instruction for students. The theory of change

undergirding this AEMDD project is as follows:

Giving school leaders and teachers of the arts access to benchmark arts assessments to

measure their students’ progress towards the Blueprint Standards, and developing professional

learning communities in these schools to use data from the assessments to determine gaps in arts

instruction and build rigorous, sequential arts programming, will lead to stronger arts

programming in the schools and better outcomes for students.

(2) SIGNIFICANCE

(a) The likely utility of the products (such as information, materials, processes, or

techniques) that will result from the proposed project, including the potential for

their being used effectively in a variety of other settings.

In response to the need for increased quality, rigor and accountability in the arts, this

AEMDD project has the following four overarching goals:


o Goal 1: To create, pilot and refine 5th

grade benchmark arts assessments in

each of the four art forms that are aligned with National, State, and

Blueprint Arts Standards.

o Goal 2: To build the capacity of teachers and school teams to deliver

Blueprint based art instruction, meet NYSED arts standards and instructional

requirements in their upper elementary school arts programs based on the

results of the assessments, through units of study, formative assessments, and

intensive professional development for school leaders and teachers.

o Goal 3: To improve students’ performance in the arts, in their 21st Century

learning skills, and in their achievement in ELA and math.

o Goal 4: To share the tools and lessons learned through the AEMDD grant

project with NYC and the larger education community.

The benchmark arts assessments will be developed in Year 1 by a specially developed

Arts Assessment Development (AAD) Team, comprised of OASP Arts Directors; staff from

cultural partners; elementary school teachers; and experts from the NYC DOE, including a

psychometrician and representatives from departments serving special needs students. The

assessments will be pilot tested in the winter of Year 1 with a group of 20 elementary schools

that are not part of the study (including approximately 3,000 5th

grade students). Thorough

analysis of the reliability and validity will be conducted and assessments will be refined as

necessary. Special attention will be given to adaptations that are required for use with English

language learner and special education populations.

In late spring of Year 1, the refined benchmark assessments will be used with 5th

graders

in the treatment and control schools to provide baseline information on the arts achievement of


the students in both groups. These data also will be used in the treatment schools to develop

comprehensive plans for implementation beginning in Year 2 geared toward identifying and

remedying gaps in arts instruction. Arts Achieve cultural partners will work closely with schools

through multiple year consultancies to build in-school and cross-school professional learning

communities (PLCs). The work of the PLCs will focus on building sequential arts programming

that meets Blueprint benchmark standards. The PLCs will use data from the benchmark arts

assessments to ensure rigorous and sequential instruction that will impact student achievement in

the arts, core content areas, and essential 21st Century learning skills. They also will work with

teachers to provide guidelines on the use of formative assessments in their instruction.

In the final year of the project, a key focus will be on dissemination of results. The

treatment schools will serve as model sites for low-performing arts schools, as determined by

NYC’s Annual Arts in Schools survey. Additionally, the NYC DOE will develop a plan to roll

out the 5th

grade assessments citywide, and the cultural arts partner organizations will share

findings and lessons learned with other arts organizations around the city through regular

convenings, such as Face to Face, the annual conference for the NYC Arts-in-Education

Roundtable and the CommonGround conference, New York State’s premier arts education

gathering hosted by the Alliance for the Arts.

The NYC DOE’s OASP proposes to carry out the work of this project with three of the

City’s most experienced and expert cultural arts partners: STUDIO IN A SCHOOL, ArtsConnection,

and Weill Music Institute at Carnegie Hall.

The OASP is in a unique position to be able to accomplish the goals set out for this

project, given our responsibility for overseeing arts education across all of NYC’s 1,600 schools.

Our staff includes an Executive Director; four Arts Directors, who have responsibility for


oversight of each of the art forms; two Directors of Arts Education Accountability; and a

Director of Operations. Major responsibilities of the OASP include:

Offering technical assistance directly to NYC’s schools on how to offer sequential

arts learning based on Blueprint Standards, implement effective arts instruction

and assessments that are reflective of best practices, and meet National Standards

and NYS requirements in the arts;

Supporting the professional development of nearly 3,000 teachers of the arts city-

wide, along with providing ongoing training in the arts for classroom teachers;

Providing arts leadership training for school leaders and arts liaisons on ways to

improve the quality and quantity of arts programming in the schools;

Providing schools with data on the arts programming in their schools, including

the extent to which they are meeting NYS requirements and the extent to which

students and parents are satisfied with arts offerings;

Working closely with postsecondary institutions to partner in the development of

teachers with arts certifications and to bolster the arts teacher pipeline; and

Partnering with arts cultural organizations across the city to strengthen art

programming within the schools.

Some highlights of our recent accomplishments include:

The development of an Arts Education Toolkit for School Leaders. The toolkit

includes all four Blueprints and charts which, as described earlier, identify the

scope and sequence of arts learning; a Quality Arts Education in NYC DVD, and a

copy of the Arts Education Manual: A Blueprint for School Leaders, which is a


guide to resources available to principals for implementing and sustaining quality

school arts programs.

Development and dissemination of Annual Arts in Schools Reports, which

include both system-wide and individual school reports and provide information

about the state of arts education in aggregate and on an individual school level

(see the Attachments for an example of an individual school report).

Creation of an Arts Reflection Tool for school leaders, which addresses both

instructional practice and programmatic structures to be used by school leaders,

arts organizations, teachers, and teaching artists in the design of arts education

programming.

Design and dissemination of High School Commencement Exams in all four

disciplines, which ensure a consistent and rigorous course of study across all of

the city’s high schools. These exams allow students who have taken a three- and

five-year sequence and passed this test to graduate with a Chancellor’s Endorsed

Diploma in the Arts. (See Attachments: Twelfth Grade Commencement

Assessment Descriptors)

As is implicit in our role within the NYC DOE, we are in a unique position to use the

knowledge and tools that we develop across all of NYC’s schools. The 5th

grade benchmark arts

assessments will be rolled out across the city following this grant period, resulting in

incorporation of the results into the data-systems for all of NYC elementary schools. The

Deputy Chancellor for the Division of Teaching and Learning, Santiago Taveras, has expressed

support for this project (see the Attachments for letter of support from Mr. Taveras). When

rolled out citywide, data from the 5th

grade benchmark arts assessments can influence policy at


the city level and decision making about arts instructional programs and resources for the arts at

the school level. Therefore, the federal funding for this project has the potential to create seismic

change across NYC, resulting in a transformational impact on the delivery of quality, sequential

arts instruction and increased and measurable student achievement in the arts.

Moreover, impact from this project may be seen not only in NYC, but also across New

York State (NYS), as the city is a leader in innovations in art education in the state. The State

Education Department’s Basic Education Data System (BEDS) arts appendix was crafted to

parallel the NYC DOE ArtsCount survey questions. Paul King, OASP’s Executive Director,

maintains steady and ongoing communication with state personnel responsible for the arts. With

regard to the current proposal, the team has already met with the Senior Deputy Commissioner of

Education, P-12 for New York State, John King, and briefed him about the plans to develop the

5th

grade benchmark arts assessments. This initial briefing was well accepted by the NYSED

and discussion was initiated in making these assessments available state-wide following the pilot

and NYC implementation and analysis. These assessment tools will prove particularly valuable

across the State, as other districts do not have the staff or capacity to create their own valid

assessments. (See Attachments for letter of support from NYSED Deputy Commissioner, John

King.)

As mentioned earlier, in order to enrich our work and broaden the reach of this AEMDD

project, we have engaged three of the city’s premier arts education organizations, with whom we

have longstanding and successful partnerships, to work with us on this grant. Our partners,

STUDIO IN A SCHOOL, ArtsConnection, and the Weill Music Institute (WMI) at Carnegie Hall,

each bring unique skills and qualifications to the table that will enhance the capacity of our

project. Descriptions of each of the partners follow below:


STUDIO IN A SCHOOL (STUDIO) Recognized as a USDOE arts-in-education model

developer, STUDIO was founded in 1977 and is the oldest and largest arts-in-education, nonprofit

organization in New York City dedicated solely to visual arts. To provide students with a full

and meaningful art experience, STUDIO links NYC public schools with local artists who draw

from their own professional expertise, artistic acumen and aesthetic sensibilities to implement

broad, performance-based art programs for students, grades pre-K through 12. STUDIO is also a

major resource for professional development programs, and encourages teacher collaboration

through multiple planning sessions and strong, long-term relationships with sites. STUDIO serves

approximately 35,000 students and 1,600 teachers annually, in more than 140 program sites

throughout New York City. Thomas Cahill, STUDIO’s President and CEO, served as co-chair,

with the DOE’s Director of Visual Arts, of the committee that created the Blueprint for Teaching

and Learning in the Visual Arts.

ArtsConnection. Since it was founded in 1979, ArtsConnection has been recognized as

a USDOE arts-in-education model developer and has been responding to the needs of the New

York City public school system, creating comprehensive programs to support its core belief that

the arts are essential for children’s learning and development. The organization continually

refines strategies to meet changing educational needs, bringing depth and diversity to its artist

residencies, family and after-school programs in music, dance, theater and the visual arts. To

maximize the success of programs and build capacity for the arts in education, ArtsConnection

also provides extensive professional development for school administrators, classroom teachers,

arts specialists, and teaching artist faculty. ArtsConnection puts a high priority on evaluation and

has conducted research and evaluation over the past twenty years to advance knowledge of the

impact of arts learning on student achievement and to share best practices with the field. The


organization directs its work to the city’s high needs communities and each year reaches 30,000

students (grades P-12) in 100+ public schools across the five boroughs.

Weill Music Institute (WMI) at Carnegie Hall. WMI creates broad-reaching music

education and community programs that play a central role in Carnegie Hall’s commitment to

making great music accessible to as wide an audience as possible. Woven into the fabric of the

Carnegie Hall concert season, these programs occur at Carnegie Hall as well as in schools and

throughout neighborhoods, providing musical opportunities for everyone, from preschoolers to

adults, new listeners to emerging professionals. The Weill Music Institute’s school and

community programs annually serve over 115,000 children, students, teachers, parents, young

music professionals, and adults in the New York metropolitan area and across the US, as well as

65,000 people around the world through its online and distance learning initiatives. In addition to

its broad-reaching education programs, Carnegie Hall also is committed to sustained work in

NYC public schools. In 2008, WMI began a multi-year, in-depth partnership with a NYC public

school in order to provide full K-8 implementation of sequential and comprehensive music

programs, in alignment with the New York City Blueprint for Teaching and Learning in the Arts,

as well as State and National Standards. The goal of this partnership is to create a model public

school with rich curriculum resources, innovative teaching practices, and assessment tools that

other arts organizations and schools can use and adapt.

(3) QUALITY OF THE PROJECT DESIGN

(a) The extent to which the design of the proposed project reflects up-to-date knowledge

from research and effective practices.

The past decade of research has provided strong evidence of the benefits of the arts on

student development, including student attitudes, attendance, behavior, motivation, and


engagement (see, for example, Minneapolis Department of Education, 2007; Los Angeles

County Arts Commission, 2004; and Burton et al., 2000; Ruppert, 2006). Recently, educators

such as Elliot Eisner (2002, 2004) also have focused on the benefits that the arts have on the

encouragement of certain 21st Century learning skills1 that are essential to success in the modern

world. It is clear that to be prepared for the complex work and life environments that we face in

the 21st Century, we must focus on skills such as creativity, critical thinking, communication, and

collaboration (the 4C’s of 21st Century learning skills). These skills, which are recognized by

the Obama Administration as essential for preparing students for the future, are fostered through

the arts and critical for success in all core content areas. Evidence abounds of the effect of the

arts on 21st Skills, such as problem solving abilities, critical thinking, elaboration, decision

making, creative thinking, and verbal/written expression (Burton et al., 2000; Catterall &

Waldorf, 1999; Eisner, 1998; Horowitz, 2005; McCarthy et al., 2004; Ruppert, 2006; Winner &

Hetland, 2007).

Yet, despite rich evidence of the benefits of a strong arts program, the arts are often

pushed to the side, while other core content areas take precedence in education as primary tools

for America’s future workforce (Eisner, 2004; Fowler, 1996). The arts frequently are the first

programs to be eliminated when budgets decrease. A key reason for this is that schools and

districts are not held accountable for student achievement in the arts because they do not have

access to quality, authentic assessments. As Fowler (1996) points out, in order for the arts to

gain respect as a core subject, ―it must look and act like one.‖

While it is clear that assessments in the arts are necessary in order to ensure that the arts

are valued and given the necessary time and resources they need, it would not be productive to

1 21

st Century Skills is defined according to the Framework of 21

st Century Skills (Partnership for the 21

st Century

Skills, 2009).


make mistakes that have been made in assessing other content areas. It is absolutely essential

that a comprehensive assessment program is used—one that balances both formative and

summative student learning and achievement data and provides information on where students

are relative to arts learning targets and standards.

Current thinking on the effective use of assessments emphasizes the need for teachers to

take ownership of constructing the tools in order to provide descriptive feedback on student

needs and to empower students to become reflective learners and take ownership of their own

learning (Reeves, 2005). Stiggins (2010) argues the importance of using a balanced assessment

system, writing: ―For teachers, competence in the classroom centers on the ability to build

quality assessments that yield accurate information about student achievement and the ability to

use the classroom assessment process and its results, not merely to monitor learning, but to

enhance it.‖ Gewertz (2010) expands on this, writing that effective assessments should

―integrate results into data systems to guide instruction and be well-integrated with curriculum

and professional development.‖

Based on knowledge of best practices in the use of assessments, this project intends to

use both formative and summative assessments to inform instruction and to make a positive

impact on student learning. The 5th

grade benchmark arts assessments will provide summative

data on students’ performance relative to arts standards. Formative assessments will be

incorporated by teachers as part of the units of study. Guidance on the use of formative

assessments will be an important part of the work that the cultural arts partners do with the

professional learning communities (PLCs) in each of the schools. The PLCs will then be

instrumental in ensuring that the formative assessments are developed and used appropriately in

the 3rd

-5th

grade classrooms.


Implementing effective formative assessments is just one of the important tasks that the

PLCs will have in their schools. Developing PLCs is a cornerstone of the Arts Achieve project.

This project draws on the most current thinking around professional development for educators,

which emphasizes the need for PLCs to develop the collective capacity of staff to work together.

According to DuFour (2005), ―The powerful collaboration that characterizes professional

learning communities is a systematic process in which teachers work together to analyze and

improve their classroom practice. Teachers work in teams, engaging in an ongoing cycle of

questions that promote deep team learning. This process, in turn, leads to higher levels of

student achievement.‖ PLCs are highly effective vehicles for professional development because

teachers learn from each other as part of their work. In order to be effective, learning must be

both school based and job embedded (DuFour, Eaker, & DuFour, 2005).

(b) The extent to which the proposed project is part of a comprehensive effort to

improve teaching and learning and support rigorous academic standards for

students.

The overarching goal for the Arts Achieve project is to improve accountability for student

learning in the arts and, in doing so, increase the number of arts instructional hours; the quality

of the instruction; and student outcomes related to the arts, 21st Century learning skills, and core

content areas, including ELA and math. The chart below summarizes the specific goals,

objectives, and outcomes that have been established for the project. The evaluation section

provides a detailed description of the proposed evaluation methods and measures that will be

employed to assess the extent to which the objectives and outcomes have been achieved.


Arts Achieve: Impacting Student Success in the Arts

Project Goals, Objectives and Outcomes

Goal 1: To create, pilot and refine 5th

grade benchmark arts assessments in each of the four

art forms that are aligned with National, State, and Blueprint Arts Standards.

Process Objectives: Outcomes:

P1.1 By the end of the planning year, an Arts

Assessment Development (AAD) Team will

develop and pilot test 5th


assessments in each of the four art forms.

P1.2 In each year of the project, the OASP will

provide training to adjudicators/scorers in the

appropriate administration and scoring of the

benchmark assessments.

O1.1 In the spring of Years 1-4 of the project,

refined 5th

grade benchmark assessments with

established reliability and validity will be used

in each of the treatment schools to inform arts

programming and to strengthen student skills.

(P1.1)

O1.2 Following their participation in training,

administration, and scoring activities, at least

90% of test adjudicators will report that they

increased their understanding of the purpose of

the assessment, the rationale behind the scoring

scales, and ways in which the results should

best be used. (P1.2)

Goal 2: To build the capacity of teachers and school teams to deliver Blueprint based arts

instruction and meet NYSED arts standards and instructional requirements in their upper

elementary school arts programs based on the results of the assessments, through units of

study, formative assessments, and intensive professional development for school leaders and

teachers.


P2.1 In each implementation year of the project,

cultural partner organizations will provide on-site

consultancies for the treatment schools that focus

on identifying and filling gaps in schools’ arts

programming to ensure rigorous and sequential

O2.1 In each implementation year of the

project, treatment schools will demonstrate

greater gains in their arts instructional hours in

grades 3-5 in the art form of focus than schools

in the control condition, as measured by the


offerings for students that are aligned with

Blueprint, State, and National standards in the arts.


the OASP and its partners will host two full-day

summer institutes for participating treatment

school teams to review and analyze the results of

their benchmark assessments, continue the work of

the on-site consultancies, and form/convene cross-

site professional learning communities by art

discipline.

P2.3 In Years 1-3 of the project, the OASP and its

cultural arts partners will host extended retreats for

school staff that will focus on the appropriate use

of formative assessments.


teams from the treatment schools will participate

in three full-day intervisitations to model

demonstration sites in order to expand their

capacity to implement rigorous and sequential arts

programs.

P2.5 In each implementation year of the project, at

least 75% of participating staff from the treatment

schools will use ARIS Connect as a venue for

archiving Blueprint aligned units, for identifying

exemplar units and formative assessments, and for

sharing best practices and lessons learned across

the project.

data from the Annual Arts in Schools survey.

(P2.1-P2.5)

O.2.2 In each implementation year of the

project, at least 90% of members of the

treatment schools’ professional learning

communities will report enhanced knowledge

of the Blueprint standards, understanding of the

arts needs at their school, and knowledge and

use of formative assessments. Staff in

treatment schools also will show greater

increases in these areas across the four years of

the grant than will staff in the control schools,

as measured by annual pre/post surveys. (P2.1-

P2.5)


project, treatment school administrators will

report a greater focus on gaps in arts

programming, better understanding of how to

fill the gaps, and more confidence in working

with cultural arts providers at their school, as

measured by annual pre/post surveys.

Administrators in treatment schools also will

show greater increases in these areas across the

four years of the grant than will administrators

in the control schools, (P2.1-P2.5)


project, participating teachers and

administrators will report that their

participation in the within-school and cross-

school PLCs has been beneficial to their

professional knowledge and skills and also has


been a factor in improving their school’s arts

programming, as measured by annual pre/post

surveys. (P2.1-P2.5)

Goal 3: To improve students’ performance in the arts, in their 21st Century learning skills,

and in their achievement in ELA and math.


P3.1 In each project year, the Arts Achieve partner

organizations will work closely with school staff to

identify, refine, and implement units of study that

are aligned with the arts skills, content and

understandings as outlined in the Blueprints. Units

will reflect alignment with 21st Century learning

skills and will include formative assessment

practices that will inform teacher instruction and

improve learning outcomes in the arts.

O3.1 In Years 2-4, it is expected that each

treatment school will increase the percentage of

5th

graders meeting arts standards and 21st

Century learning skills on the benchmark

assessments by 10% over the prior year, as

measured by the benchmark arts assessments.

(P3.1)

O3.2 By the end of Year 4, it is expected that

students in the treatment group will demonstrate

significantly better performance in their art

skills in the school’s art form of focus and on

21st Century learning skills than students in the

control schools, as measured by the 5th

grade

benchmark arts assessments. (P3.1)

O3.3 (GPRA) In each implementation year, it is

expected that students in the treatment group

will demonstrate significantly improved

performance on their ELA (specific writing

criteria include: Ideas and Content,

Organization, Voice, Word Choice, Sentence

Fluency, and Conventions) and math skills as

compared to students in the control schools, as

measured by the annual NYS ELA and Math

exams. (P3.1)


Goal 4: To share the tools and lessons learned through the AEMDD grant project with NYC

and the larger education community.


P4.1 In each implementation year, the NYC DOE

and each of its Arts Achieve cultural partners will

widely disseminate the AEMDD grant and its

work on their websites and through other venues.

P4.2 By the end of Year 4, the OASP will develop

plans to roll out final, refined versions of the 5th

grade benchmark arts assessments to elementary

schools across the city, as well as to support the

schools in their use of the assessment results.

O4.1 In Years 3 and 4, the OASP and its Arts

Achieve cultural arts partners will share results

of the study at convenings of cultural arts

organizations (such as Face to Face and

CommonGround) and at gatherings of NYC

principals (such as the NYC Summer Institute

for Principals). (P4.1)

O4.2 In Years 3 and 4, the OASP the Arts

Achieve partners and evaluation team will

submit proposals to present at a total of at least

two national arts conferences and/or meetings of

discipline based organizations (such as MENC,

NAEA, NDEO, AATE and others). (P4.1)

O4.3 In Years 3 and 4, project evaluators will

submit proposals to present findings at national

evaluation conferences, such as AERA and

AEA. (P4.1)

O4.4 In Years 3 and 4, the treatment schools

will serve as models for NYC schools that are

identified as low performing in the arts based on

their Annual Arts in Schools survey data. (P4.2)

Following is a description of the design for this project, including the activities that will

be conducted to address each of the project’s goals.


grade benchmark arts assessments in each of the

four art forms that are aligned with National, State, and Blueprint Arts Standards.


Year 1 of the project will be a planning year, during which the following activities will

take place:

Development of 5th

grade benchmark arts assessments. Arts Assessment

Development (AAD) Teams will be assembled shortly after the start of Year 1 to

spearhead the creation of the arts assessments. The teams will be led by the NYC DOE

Arts Directors in the four art forms, and also will be comprised of school-based arts

teachers, staff from the cultural partner organizations, and NYC DOE experts in tests and

measurement and in special populations. Separate assessments will be developed for

each of the four art forms: visual arts, dance, music, and theater. Each assessment with

have three distinct components: a multiple choice section, which will be comprised of

one set of questions addressing content knowledge in the particular art form and one set

of questions that cuts across general art content and concepts; a performance section,

which will assess students’ application of arts skills in the art form being tested; and a

written response section in which students will be asked to produce a response to a work

of art, discuss the work of an artist they have studied or compare two works of art within

the art form being tested. The work of the AADs will involve identifying the arts skills

and content understandings to be measured on each of the three components of the

assessments, as well as identifying 21st Century learning skills that are implicit in the

knowledge being measured. The OASP has already begun this important work and the

AAD teams will expand and enhance it through this grant. The table on the following

page shows the potential alignment between Dance Performance benchmarks and 21st

Century learning skills for Dance. Full alignments for each of the other arts forms are

included in the Attachments.


Alignment Between Dance Performance Benchmarks and 21st Century learning skills

Dance Performance Assessment

Performance Task Arts Benchmark

Alignment

21st Century learning skills

Dance Making:

Students develop

skills and techniques

Students execute a variety

of locomotor movements

with changing shapes,

dynamics, and pathways

Think creatively, solve problems,

work independently, produce

results

Dance Making:

Choreograph

Students select themes,

discuss and plan, and

develop movement in

collaboration with peers,

in partners and small

groups. Students create a

short dance with peers,

incorporating several

movement phrases with a

beginning, middle, and

end.

Think creatively, solve problems,

reason effectively, communicate

clearly, be responsible to others,

collaborate with each other,

interact effectively, make

judgments and decisions, be

flexible, work creatively with

others, manage projects, produce

results.

The NYC DOE expert in tests and measurements will provide guidance on best practices

in the construction of both multiple-choice and written response items. She also will

provide assistance in vetting Performance Task items and associated adjudicator rubrics

prior to the pilot and implementation. The NYC DOE experts in special populations

will advise on modifications to the assessment that may be required for English language

learners (ELL) and special education students.

Training for school-based test adjudicators. Immediately following the development

of the draft assessments, the OASP will provide training for school-based adjudicators in

administering and scoring the assessments. This training will be provided in January of

Year 1 for adjudicators in the pilot schools and April of Year 1 for adjudicators in the

treatment and control schools. During the training, the NYC DOE psychometrician, in


collaboration with the project evaluator, will ensure that adequate inter-rater reliability is

reached for scoring performance rubrics.

Pilot and refine 5th

grade benchmark arts assessments. The benchmark assessments

will be piloted in February of Year 1. A total of 20 NYC public elementary schools will

participate in the pilot. These schools will not be part of the pool for the study and will

be representative of the total population of NYC schools in terms of location, size,

student demographics and level of arts implementation. The NYC DOE

psychometrician, in collaboration with the project evaluator, will oversee the pilot and

analyze the psychometric properties of the tests. Performance and written components of

the assessments will be scored by the adjudicators, using rubrics that will be developed

and vetted for adequate inter-rater reliability. Performance components will be scored on

site at the schools.

The assessments will be refined in March and April of Year 1 based on the results of the

pilot and will be finalized by May of Year 1. The finalized assessments will be used in

the treatment and control schools in late May or early June of Year 1, in order to provide

a baseline for 5th

grade arts achievement at the schools. The assessments will be used in

each subsequent year in the treatment schools and, as described in more detail below, not

only will be used to assess students’ art skills but also will be used to inform strengths

and weaknesses in the arts programming. Data from the assessments ultimately will be

available on ARIS, on individual school arts reports and will become part of the data to

which school staff and parents have access.

Goal 2: To build the capacity of teachers and school teams to deliver Blueprint based arts

instruction, meet NYSED arts standards and instructional requirements in their upper



study, formative assessments, and intensive professional development for school leaders

and teachers.

Goal 3: To improve students’ performance in the arts, in their 21st Century critical

thinking skills, and in their achievement in ELA and math.

The activities described below are intended to impact on both Goals 2 and 3.

As part of the intervention, treatment school staff will participate in intensive and

customized professional development to help them strengthen the arts programming at their

schools. The professional development will include the creation of professional learning

communities (PLCs) at two levels: 1) within each treatment school; and 2) across treatment

schools by art discipline. Within each school, the members of the PLCs will include the school-

based arts teachers, the school arts liaison, grade-level teacher leaders in grades 3-5, and

representatives from any cultural arts partners that are working in the school building. This team

will participate in professional learning experiences, as described below, and will be part of

cross-school PLCs. The PLCs will provide a forum for sharing, reflecting, training, and

planning appropriate units of study that will scaffold sequential learning for students.

Professional learning experiences that will take place in Years 2-4 include the following:

Participation in on-site consultancies with one of the partner cultural arts

organizations. Starting in August 2011 and continuing through the end of the project, the

cultural arts partners will work closely with the treatment schools to assist them in strengthening

their arts programming and ensuring sequential offerings that are aligned with National, State

and Blueprint standards. Each treatment school will focus on one art form (for a total of two


treatment schools per art form). The following scheme shows how treatment schools will work

with the cultural arts partners.

Partnership Scheme for Cultural Partners and Treatment Schools2

Cultural Partner Art Form Schools

ArtsConnection Dance Schools 1 and 2

Carnegie Hall Music Schools 3 and 4

ArtsConnection Theater Schools 5 and 6

STUDIO IN A SCHOOL Visual Arts Schools 7 and 8

This consultancy work, which will take place over 12-15 full days per year at each site,

will be multi-faceted and while the specific activities will be unique to the needs of each

individual school, the following components will underlie the work:

o Each year, the cultural arts partners will work with the PLCs to analyze results from

the most recent administration of the 5th

grade benchmark arts assessments and to

determine where gaps are in current students’ knowledge and skills in the arts. They

will target the alignment of the Blueprint learning goals in their focus art form,

provide guidance in effectively using formative assessments, and work to enhance

instruction that is responsive to student needs.

o Following an analysis of weaknesses in students’ art skills, the arts partners will

gather information from multiple existing data sources (i.e., standardized tests, report

cards) about areas where students are weak in their ELA and math skills.

2 Schools must offer at least 50 hours of arts instruction in an art form to all grades 3-5 to be eligible to participate in

project activities for that art form.


o Using information gathered from analyses of students’ needs in the arts and other

core content areas, they will work to identify, refine, and cache units of study in the

arts. These units will be aligned with the standards in the arts and meet all identified

needs of the participating schools. The arts cultural partners, in collaboration with the

PLCs, will focus their efforts on the need to effectively scaffold arts instruction in

grades 3-5 to ensure students meet the standards by the end of 5th

grade.

o The Arts Achieve partners also will work with the PLCs to provide guidance on

effective ways to incorporate formative assessments into the units. While the

formative assessments will be unique according to the needs of the students and the

unit of study, the team will develop guidelines for the effective use of these

assessments to set clear learning goals for students and to provide opportunities for

them to get rich, descriptive feedback from their teacher, from their peers and from

their own self-reflection.

o An additional important component of the consultancy will be to provide guidance to

the schools in working effectively with all arts partners in their buildings. Because

the arts partners will be part of the PLCs, all individuals offering arts will be aware of

student needs and can ensure that instruction is sequential and targeted toward the

standards.

Participation in Summer Institutes. Summer Institutes will be implemented in Years 2,

3, and 4. In August 2011, at the start of Year 2, a two day summer retreat will be hosted

by the OASP and its cultural arts partners. Members of the treatment school PLCs will

be in attendance at the retreat and will use this time to begin the work that will be

continued during the in-school consultancies described above. Additionally, the four


cross-school PLCs (dance, music, theater, and visual arts) will meet and begin to share

ideas and practices. Summer institutes in August 2012 and 2013 will each be two days in

length and will include the same participant groups. These retreats will allow schools to

review the results from their spring assessments and provide them time to continue the

work that takes place in the consultancies and cross-school groups throughout the year.

Participation in multiple full-day retreats in the use of formative assessments over

the course of the project. In Years 1-3, the PLCs will participate in retreats focused on

the development and use of formative assessments in the arts (the project will support

two full days in Year 1, three full days in Year 2, and one full day in Year 3). Dr. Heidi

Andrade, a professor at SUNY Albany and an expert in the field, will facilitate these

retreats (see the Attachments for her resume). Her training will center on building school

staff’s ability to develop formative assessments and ways to use them to improve teacher

instruction and student learning. Her work will highlight the need for student

engagement in the process and the importance of the assessments in encouraging student

motivation and responsibility for learning. Both the summer institutes and the formative

assessment training with Dr. Andrade will powerfully reinforce the work of the PLCs and

their focus on student achievement.

Facilitate a series of full day inter-visitations to model demonstration sites. During

each implementation year of the grant, the OASP will facilitate a series of full-day

intervisitations, during which treatment schools will visit model schools that have been

developed by their arts partners. For example, Carnegie Hall has a model Music

Blueprint School, ArtsConnection has Model Sites in dance and theater, and STUDIO IN A

SCHOOL has model demonstration sites in Visual Arts. These visits are intended to


develop the school-based PLCs’ capacity to plan instruction, design and adapt units of

study, use formative assessment tools and strategies, and provide descriptive feedback to

students. In the intervisitation model, teachers will use a protocol that guides their

observation and discussion around student work and engagement, thus promoting a forum

for sharing, reflecting, training, and planning appropriate units of study that will scaffold

sequential learning for students.

Use technology to share best practices. In each implementation year, project

participants will use ARIS Connect, a NYC DOE social networking platform, as a venue

for the cache of Blueprint aligned units, for identifying exemplar units and formative

assessments, and for sharing best practices and lessons learned across treatment schools.

Goal 4: To share the tools and lessons learned through the AEMDD grant project with

NYC and the larger education community.

The Arts Achieve project ultimately will lead to finalized 5th


assessments that can be rolled out across NYC. Results from these arts assessments will give

schools information that they need about the quality of their arts programming and the extent to

which it is adequately sequenced and scaffolded to ensure that students meet Blueprint, State and

National Standards. By the end of Year 4, the OASP will develop a plan for this roll out and will

share results with the NYC DOE and NYSED.

In addition, the OASP and its cultural arts partners will feature the work of Arts Achieve

on their websites and will disseminate results at annual convenings of the cultural arts

organizations (e.g., Face to Face and CommonGround), as well as at local, regional, and national

conferences (e.g., NYC and NYS Association of Arts Teachers and NYC Arts in Education

Roundtable), and at gatherings of principals (the NYC Summer Principal Institute). The


evaluators, in collaboration with project personnel, will present findings at national conferences

on educational evaluation, such as AERA and AEA.

(c) The extent to which the proposed project is designed to build capacity and yield

results that will extend beyond the period of Federal financial assistance.

As described above, the goals of the AEMDD project address the USDOE’s requirements

that grantees’ efforts result in increased capacity and extend beyond the grant period. The

OASP, along with its cultural partner organizations, will work closely with the treatment schools

to develop PLCs that will focus their work on building their schools’ capacity to implement

rigorous and sequential arts programs that are aligned with Blueprint standards and emphasize

necessary 21st Century learning skills.

Additionally, Goal 4 of the project focuses specifically on the dissemination and

replicability of the grant. The OASP and its partners will use technology and conference

opportunities to disseminate findings from the project. In Year 4 and the years following the

grant, the treatment schools will serve as dissemination sites for other schools around the city.

They also will function as model sites for schools that are identified as low performing in the arts

based on the results of their Annual Arts in Schools survey data.

(4) QUALITY OF PROJECT PERSONNEL

(a) The qualifications, including relevant training and experience, of key project

personnel.

Paul L. King, the Executive Director of the OASP at the NYC DOE, will serve as the in-

kind Project Director for the AEMDD project. In this role, Mr. King will supervise the Project

Manager, serve as the head of the Arts Achieve Steering Committee (AASC) and will assure that


the work of this initiative aligns with other NYC DOE programs and policies. Additionally, Mr.

King will be the primary liaison to NYSED to keep them informed about the project’s progress.

Prior to being named Executive Director of OASP, Mr. King served for four years as the

NYC DOE’s Director of Theater Programs. In this role he developed the Blueprint for Teaching

and Learning in the Arts: Theater, supervised and implemented the creation of 12th

Grade

Commencement Assessments in Theater, and managed a nationally normed assessment in

Technical Theater in conjunction with the National Occupational Testing Institute (NOCTI).

Before coming to the NYC DOE, Mr. King was the Director of Education and

Community Service for New York City Opera—a position he held for seven years. Mr. King has

served as an artist educator for various opera companies and theaters in the United States with

over 12 years experience in high school, middle school and elementary school instruction.

Additionally, Mr. King has been a teacher, stage director and guest artist for singer training

programs at Chautauqua Opera, The Glimmerglass Opera, The Manhattan School of Music and

SUNY/Purchase. Mr. King was the recipient of the 2008 Broadway Theatre League’s Apple

Educator Award and is a member of the Mayor’s Task Force on Diversity in Film, Television,

and Commercial Production in New York City. Mr. King also served as a member of the New

York City Board of Education’s Pre-Professional Advisory Committee for Performing and

Visual Arts and as a grant review panelist for the Surdna and Doris Duke Foundations on the

Talented Students in the Arts initiative. Formerly, he was a member of the Center for Arts

Education Parents as Arts Partners advisory board. Mr. King holds a B.A in Theater from the

Colorado College and a M.F.A. in Performing Arts Management from Brooklyn College.

Mr. King will also be supported in this project by a Project Manager, who will function

as the principal liaison between OASP (see full job description in the Quality of the Management


Plan section), school staff and principals, and arts partners. In addition, several other members

of the OASP Executive Team will provide significant support to the project, including the four

OASP Arts Directors, Karen Rosner (Coordinator of Visual Arts), Barbara Murray (Director

of Music Programs), Peter Avery (Director of Theater), and Joan Finkelstein (Director of

Dance Programs), and the co-directors of Arts Education Accountability and Support, Eileen

Goldblatt and Maria Palma. Resumes for these key OASP staff members are provided in the

Attachments.

Denise Mutlu, Director of Test Design and Development from the NYC DOE’s Policy

and Research Group, will support the design of the 5th

grade benchmark arts assessments. Ms.

Mutlu will work closely with other members of the AAD Team to develop multiple choice,

performance, and written test items for the 5th

grade assessments; analyze results from the pilot

of the instrument; and provide consultation on item or other test revisions. Ms. Mutlu’s resume

is provided in the Attachments.

Finally, the project directors at each partner organization will play a critical role in the

effective implementation of the project. Resumes for Tom Cahill (STUDIO IN A SCHOOL),

Lauren Collins (Arts Connection) and Amy Kirkland (Carnegie Hall) are provided in the

Attachments.

(b) In determining the quality of personnel, the extent to which the applicant

encourages applications for employment from persons who are members of groups

that have been traditionally underrepresented.

The New York City Department of Education’s policies for non-discrimination in its

hiring practices are articulated and safeguarded by its Office of Equal Opportunity. These

policies are in accordance with the following federal and state laws:


Title VI and Title VII of the Civil Rights Act of 1964, as amended;

Title IX of the Education Amendments of 1972;

the Age Discrimination in Employment Act of 1967, as amended;

Section 503 and Section 504 of the Rehabilitation Act of 1973, as amended;

the Fair Labor Standards Amendments of 1974;

the Americans with Disabilities Act of 1990, as amended;

the Civil Rights Act of 1991;

the New York State Human Rights Law; and

the New York City Human Rights Law and Provisions of Non-Discrimination in

Collective Bargaining Agreements of the Board of Education of the City School District

of the City of New York.

All New York City public school personnel are recruited in accordance with employment

procedures of the New York City Public Schools and agreed upon with the United Federation of

Teachers (UFT), the union that represents staff positions in collective bargaining. The NYC

DOE is an affirmative action employer. The Equal Employment Opportunity/Affirmative Action

Program (EEO/AA) of the NYC DOE is designed to guarantee that ―emphasis will be given to

identifying and selecting minorities and women from the qualified pool for positions where they

are underutilized and to making efforts to increase the representation of minorities and women in

the qualified pool.‖

(5) QUALITY OF THE MANAGEMENT PLAN

(a) Adequacy of the management plan to achieve the objectives of the proposed project

on time and within budget, including clearly defined responsibilities, timelines, and

milestones for accomplishing project tasks.


(b) The extent to which the time commitments of the project director and principal

investigator and other key personnel are appropriate and adequate to meet the

objectives of the proposed project.

As noted in Section 4, oversight for the proposed AEMDD project will be the

responsibility of a Project Manager, who will ensure the timely, smooth and effective

implementation of all project components. The Project Manager, who will spend 40% of his/her

time on this project, will report directly to Paul King and will have overall programmatic and

administrative responsibility for the project. The Project Manager will be responsible for

carrying out the following tasks:

providing technical assistance to the treatment schools in program implementation;

representing the project at all AEMDD grantee meetings and actively participating in

other dissemination activities to share lessons learned from the project;

serving as the liaison to the cultural partner organizations, ensuring that all efforts and

supports to the treatment schools are well coordinated;

monitoring all project expenditures to review compliance with all fiscal requirements and

regulations; and

serving as the liaison to the project evaluator, assisting in the collection of data from both

the treatment and control schools and preparation of all performance and evaluation

reports.

The AEMDD project management plan also includes an Arts Achieve Steering

Committee (AASC), to be comprised of the key OASP project staff, senior staff from the three

partner organizations, and the evaluators. The work of the Steering Committee, which will be


guided Paul King, will serve to guide and inform the implementation of the project and the

rigorous evaluation, and will be convened on a quarterly basis.

The supplementary resources needed to develop and carry out the project activities are

detailed in the attached budget (ED 524 form) and budget narrative. We believe that the funds

requested are both reasonable and sufficient to realize the project's full potential for meeting its

stated objectives and outcomes. All possible efforts have been made to minimize the costs of

this project and we believe that the costs are reasonable in terms of the target population of staff

and students to be served. At the same time, the project is designed so that the achievement of

the objectives is promoted through comprehensive planning and coordination of activities that

will insure the appropriate intensity of program effort.

Although there is no cost sharing requirement for this program, as highlighted in the

description above and table below, the in-kind contributions being provided by the NYC DOE,

the OASP, the cultural partners and the participating schools will ensure that the program

operates efficiently and appropriately and that it achieves both its process objectives and desired

outcomes.

In-Kind Resources in Support of AEMDD Objectives

Organization Nature of In-kind Support

NYC DOE

Office of Arts and

Special Projects

(OASP)

Instructional leadership, technical assistance from senior staff

Dissemination activities (e.g., seminars for school leaders, arts and

cultural service fairs)

NYC DOE

Office of Student

Performance and

Accountability

Pyschometrician to oversee pilot of 5th


assessments

Special Education and ELL experts to provide guidance on

modifications needed to the assessment for special populations

Analysis of results of assessments

Cultural Partners Leadership from Directors and other senior staff for design and


Organization Nature of In-kind Support

(STUDIO IN A

SCHOOL, Carnegie

Hall, Arts

Connection)

implementation of project

Support for dissemination activities

Treatment Schools

Instructional leadership and support from administrators for project

implementation

Instructional services delivered by classroom and arts teachers and other

teacher specialists

Project Timeline

The timeline on the following pages presents major milestones for first two years of the

AEMDD project. A summary of implementation milestones for Years 3-4 follows.


Year 1

Major Project Activities Milestones Timeline Person(s) Responsible

Convene Steering Committee

Develop planning year calendar

Define roles and responsibilities

Identify and form AAD Teams

Coordinate cache of units of study

Develop professional development arc

Monthly

(July 2010-June 2011) Steering Committee

Convene Arts Assessment

Development (AAD) Teams

Develop four 5th grade benchmark

arts assessments

2x/month

(Sept 2010-Jan 2011) AAD Teams

Finalize evaluation design

Develop all evaluation instruments

Submit proposal to NYC DOE

proposal review committee

November 2010 Evaluator

Train adjudicators/raters in pilot

schools

Provide training in administration and

scoring of assessments for

adjudicators in pilot schools

January 2011 OASP Team

Pilot 5th grade benchmark arts

assessments

Conduct pilot, score and analyze

results Feb–March 2011 AAD Teams

Train adjudicators/raters in

treatment and control schools




April 2011 OASP Team

Conduct retreat on formative

assessment for Arts Achieve

partners and OASP team

Build understanding of how to use

formative assessments

Allow time for cross-team PLCs to

meet

May 2011

(2 days) Dr. Heidi Andrade

Administer and score 5th grade

benchmark exams in treatment

Collect baseline data on student art

performance in treatment and control May/June 2011 OASP Team



and control schools schools

Provide final evaluation report

Provide data on effectiveness of

planning activities and make

recommendations for future years

July 2011 Evaluator

Year Two


Convene Steering Committee

Review Year 1 evaluation findings

Analyze data from administration of

5th grade benchmark assessments

Design professional development

Coordinate cache of units of study

Monthly

(July 2011-June 2012) Steering Committee

Implement pre evaluation

instruments

Collect baseline data on participants’

knowledge and skills August 2011 Evaluator

Summer Institute Kick Off

Reflect on data from assessment

Begin developing school level plans

Develop cross-school PLCs

August 2011 OASP/Partnership Team

Conduct on-site consultancy

Build capacity of principals and

school administrators

Work closely with in-school PLCs to

analyze data, plan instruction,

design/adapt units of study, use

formative assessment, provide

feedback to students

Sept 2011-June 2012 Cultural Arts Partners

Share practices and learnings Upload exemplary units of study and Sept 2011-June 2012 OASP/Partnership Team



through ARIS Connect formative assessment tools

Build cross-school PLCs

Conduct retreat on formative

assessment

Foster understanding of how to use



meet

Oct 2011

(2 days)

Dr. Heidi Andrade

Conduct intervisitations Visit model sites and cohort schools Oct 2011-May 2012 In-School PLCs

Train adjudicators/raters in

treatment schools




Feb 2012 OASP Team

Conduct mid-winter full day

formative assessment retreat

Build understanding of how to use



meet

Feb 2012

(1 day)

Dr. Heidi Andrade

Administer and score 5th grade

benchmark exams in treatment

schools

Collect data on student art

performance in treatment and control

schools

May 2012 OASP Team

Administer post evaluation

instruments

Collect end-of-year data on

participants’ knowledge and skills May 2012 Evaluator

Provide final evaluation report

Provide data on effectiveness of

planning activities and make

recommendations for future years

July 2012 Evaluator


Project activities in Years 3 and 4 will be similar to those in Year 2. Each year

will begin with Summer Institutes that will allow the in-school PLCs time to review data,

plan for the upcoming year, and meet in their cross-team PLCs. The Steering Committee

will meet monthly throughout the year to review project data and plan activities. The

cultural arts partners will continue the consultancies in the treatment schools, meeting

with the PLC teams for 12-15 full days throughout the year. This time will be spent

reviewing gaps in programming, identifying units of study, providing guidelines for

formative feedback, and building leadership capacity in the building. A final retreat

designed to deepen the training on formative assessment for treatment schools will be

facilitated by Dr. Heidi Andrade in Year 3 (one full day). Training for adjudicators/raters

will be provided in each year and the 5th

grade benchmark arts assessments will be

administered in May of each year in the treatment schools and May of the final year in

the control schools. Additional time in Years 3 and 4 will be spent on dissemination

activities, such as presenting findings at local and national conferences and developing

plans for roll out of the assessments.

(c) The adequacy of procedures for ensuring feedback and continuous improvement in

the operation of the proposed project.

As described earlier in this section, several mechanisms will be put into place to ensure

that timely feedback is gathered, discussed and utilized to support effective project

implementation and continuous improvement:

The project management team, which will consist of the Project Director, the Project

Manager and other key OASP staff, will meet on a monthly basis to discuss the process

of implementation and collaborate on developing solutions to challenges as they arise.


The Arts Achieve Project Steering Committee (AASC), which will meet monthly, will

ensure coordination across the various project activities and alignment of all internal and

external sources in support of project objectives.

Quarterly formative evaluation meetings, which will be planned and scheduled with the

project evaluation team from Metis. One of the primary purposes of these meetings will be to

share findings from the various formative evaluation activities (e.g., observations, interviews).

In addition, Metis will provide an interim report on the findings of the fall data collection

activities each year. Thus, project staff and key stakeholders will not have to wait until a formal

report is issued before they can utilize the findings to guide the project.

(6) QUALITY OF THE PROJECT EVALUATION

This section describes the evaluation design for the proposed project, including the plan’s

methodological approaches. The NYC DOE proposes to retain Metis Associates, an independent

research and evaluation consulting firm based in NYC, to develop and conduct the project

evaluation, which includes an experimental design component. Metis has extensive experience

using experimental, quasi-experimental and other rigorous evaluation designs to evaluate

academic enrichment programs, arts education programs, and professional development

initiatives at the elementary school level in school districts around the country. Metis is

currently evaluating two AEMDD grant projects, Honoring Student Voices (Global Writes) and

Framing Student Success (STUDIO IN A SCHOOL), both of which use an experimental design to

compare outcomes for students participating in program activities to those for students in a non-

participating control group. Among the resources that Metis will make available to the

evaluation of this project is a professional staff with extensive and diverse expertise and

experience in research design, instrumentation and measurement, psychometrics, and qualitative


and quantitative research methodologies. Dr. Susanne Harnett, Managing Senior Associate, will

serve as the Principal Investigator for the evaluation. (See the Competitive Priority section for

additional information about Metis Associates, including recent project summaries and

credentials.)

(a) The extent to which the methods of evaluation include the use of objective

performance measures that are clearly related to the intended outcomes of the

project and will produce quantitative and qualitative data to the extent possible.

The proposed evaluation will be both formative and summative and will employ multiple

qualitative and quantitative methods and sources of data. An important component of the

evaluation plan is the experimental design, which is fully discussed in the Competitive

Preference Priority section of this proposal. This design will assess whether implementation of

benchmark arts assessments – when accompanied by targeted and intensive professional

development for school staff and rigorous, sequential instruction for students – leads to improved

school-wide and student-specific outcomes. The following table describes the data sources and

evaluation methods that will be used to measure progress toward meeting each of the project’s

objectives and outcomes.

Data Sources and Timeline for Evaluation of Project Outcomes

Project Outcome Data Source Timeline

Goal #1: To create, pilot and refine 5th

grade benchmark arts assessments in each of the

four art forms that are aligned with Blueprint, State, and National Arts Standards.

O1.1 Refined assessments

used in treatment schools Project Documentation Winter/Spring Years 2-4

O1.2 Test adjudicators

increase understanding of

assessment and scoring

Adjudicator Survey Winter/Spring Years 1-4

Goal #2: To build the capacity of teachers and school teams to deliver Blueprint based arts

instruction and meet NYSED arts standards and instructional requirements in their upper



study, formative assessments, and intensive professional development for school leaders

and teachers.

O2.1 Greater gains in arts

instructional hours in

treatment over control schools

Annual Arts in Schools Survey Spring Years 2-4

O2.2 Greater increases in

knowledge and understanding

among teachers in treatment

over control schools

Pre/Post Locally Developed

Surveys

Focus Groups

Workshop Feedback Forms

Fall & Spring Years 2-4

O2.3 Greater increases in

school administrator focus on

the arts in treatment over

control schools


Surveys

Interviews

Workshop Feedback Forms

Annual Arts in Schools Reports


O2.4 Increased knowledge

and skills among members of

the professional learning

communities and improved

arts programming


Surveys


Goal #3: To improve students’ performance in the arts, in their 21st Century learning

skills, and in their achievement in ELA and math.

O3.1 Increase in 5th

graders in

treatment schools meeting arts

standards and 21st Century

learning skills

5th

Grade Benchmark Arts

Assessments Spring Years 1-4

O3.2 Significant differences

between treatment and control

groups in meeting arts

standards and 21st Century

learning skills

5th

Grade Benchmark Arts

Assessments Spring Years 1-4

O3.3 Significant differences

between treatment and control

groups in ELA and Math skills

NYS ELA and Math Tests Spring Years 1-4

Goal #4: To share the tools and lessons learned through the AEMDD grant project with

NYC and the larger education community.

O4.1 Share results at

convenings of cultural arts

partners and NYC principals

Project Documentation Spring Years 3-4

O4.2 Submit national arts

conference proposals Project Documentation Spring Years 3-4

O4.3 Submit national

evaluation conference

proposals

Project Documentation Spring Years 3-4

O4.4 Treatment schools serve

as models Project Documentation Spring Years 3-4


The following paragraphs further describe the data sources and evaluation methods that

will be used to measure progress toward meeting each of the project’s objectives and outcomes.


grade benchmark arts assessments in each of

the four art forms that are aligned with National, State, and Blueprint Arts Standards

(music, art, theater and dance). To assess the process objectives of Goal 1, Metis will review

project documentation, including AAD Team meeting agendas, attendance sheets, assessment

development materials, and training guides.

Metis will conduct observations of a sample of meetings of the AAD Team (in Year 1)

and project activities at the treatment schools (in Years 2 through 4) to learn more about the

development, refinement, and use of the benchmark arts assessments (Outcome 1.1). In addition,

Metis will develop and administer a survey to test adjudicators to assess the extent to which they

understand the purpose of assessment and best uses for results after their participation in training,

administration, and scoring activities (Outcome 1.2).

Goal 2: To assist schools in meeting Blueprint arts standards and instructional

requirements in their upper elementary school arts programs based on the results of the

assessments, through units of study, formative assessments, and intensive professional

development for school leaders and teachers. To assess the process objectives of Goal 2, each

year, Metis will review project documentation, including formative assessment tools; evidence of

student learning posted on ARIS; and meeting agendas, handouts, and attendance rosters.

Additionally, Metis will conduct observations of a sample of the consultancy days and will

observe and participate in the Summer Institute and Formative Assessment retreats. Workshop

feedback forms will be created and analyzed to gather participants’ feedback on the usefulness of

the sessions.


In order to assess progress toward outcomes for Goal 2, data from the Annual Arts in

Schools Report will be used to assess changes in number of total instructional hours in arts in

treatment schools relative to control schools (Outcome 2.1). Additionally, Metis will develop

school administrator and teacher surveys in Year 1 in collaboration with OASP staff and will

administer these in the treatment schools each spring beginning in Year 2. Teacher surveys will

use a combination of Likert scale and open-ended questions to assess the extent to which

participants have increased their knowledge of Blueprint standards, the needs and gaps in the

arts at their school, and best practices for using formative assessments (Outcome 2.2). The

school administrator survey (for principals and other administrators) will assess the impact of

the project on their abilities related to understanding the results of 5th


assessments; identifying gaps in arts programming and implementing changes; and providing a

comprehensive arts program by working effectively with arts providers. Additionally, the

evaluators will review principal narrative on the Annual Arts in Schools reports to assess

whether there are changes in their responses that suggest better understanding of arts standards

and requirements (Outcome 2.3). The surveys of PLC team members also will assess their

perceptions of the value of their collaborative work throughout the project years (Outcome 2.4).

Beginning in the spring of Year 2, the evaluator also will conduct annual interviews and

focus groups with school administrators and participating teachers at the treatment schools to

gather additional qualitative data on the schools’ arts program, and at the control schools to

determine whether they are implementing any other interventions that may also have had an

impact on the results. Metis will use semi-structured interview and focus group protocols that

will be developed in Year 1 in collaboration with OASP staff.


Goal 3: To improve students’ performance in the arts, in their 21st Century critical

thinking skills, and in their achievement in ELA and math. To assess the process objectives

for Goal 3, Metis will review program documentation of cultural partner organizations’ unit

refinement efforts (Year 1) and of their work with school-based staff to further develop, refine,

and cache units of study and identify and use assessment data (Years 2 through 4). Metis will

conduct school walkthroughs in a sample of the treatment schools at periodic intervals in each

year of the project, using the NYC DOE Arts Reflection Tool, to assess the extent to which

Standards aligned, sequential arts programming is reflected in the building.

Data from 5th

grade benchmark arts assessments will be used to measure student

performance in art skills, content and understanding in the treatment schools’ art form of focus

relative to that of students in control schools. The benchmark assessments will be piloted in the

winter of Year 1 and refined according to data gathered through the pilot. Reliability and

validity will be determined through the pilot, including inter-rater reliability for all writing and

performance rubrics. The assessments will be implemented in the spring of each year of the

project in the treatment schools and in the spring of Year 1 (baseline) and Year 4 (post) in the

control schools. Data from the finalized assessments will provide information about students’

progress toward meeting standards in the arts as well as in their 21st Century learning skills (e.g.,

critical thinking, problem solving, innovation, creativity). It is expected that progressive cohorts

of 5th

grade students in the treatment schools will perform better each year in their arts and 21st

Century learning skills (Outcome 3.1). It is also expected that by the end of Year 4, 5th

grade

students in treatment schools will obtain higher scores on the benchmark assessments than

control students (Outcome 3.2), and that these differences will be statistically significant (p <


.05) and educationally meaningful (Cohen’s d > .33), as determined by analyses of covariance

(ANCOVA).

Student achievement in English language arts and mathematics will be assessed through

the New York State English language arts (NYS ELA) and mathematics (NYS Math) tests. The

NYS ELA and NYS Math exams are criterion-referenced tests that are directly aligned with State

standards and are administered each year to students in grades 3 through 8. They have proved to

be valid and reliable measures of student achievement. Results are expressed in continuous scale

scores and performance levels. By the end of Years 2 and 3, it is expected that students in

treatment schools will obtain higher mean ELA and mathematics end-of-year scale scores (after

statistically controlling for differences in pretest scores) than control students at each grade level,

and that these differences will be statistically significant (p < .05) and educationally meaningful

(Cohen’s d > .33), as determined by analyses of covariance (ANCOVA). Since baseline data are

not available for 3rd

grade students (the NYS tests are not administered in grade 2), independent

samples t-tests will be conducted for this group of students (Outcome 3.3).

Goal 4: To share the tools and lessons learned through the AEMDD grant project

with NYC and the larger education community. One of the main goals of the project will be

to share the arts assessments, related curriculum units, and relevant resources with the larger

education community. To assess this goal, Mets will review project documentation pertaining to

plan for roll-out of the final 5th

grade benchmark arts assessments to NYC schools, as well as

plans to support schools in the use of the assessment results. Documentation also will be

collected of participants’ utilization of the ARIS Connect system to communicate and share

learnings. Additionally, in Years 3 and 4, Metis will collect documentation pertaining to efforts

to share the assessments and findings within and outside of NYC, including presentations at


convenings of cultural arts organizations and NYC principals (Outcome 4.1), national and

discipline based conferences (Outcome 4.2), and national evaluation and research conferences

(Outcome 4.3). Documentation also will be collected on the use of treatment schools as models

for low performing arts schools across NYC (Outcome 4.4).

(b) The extent to which the methods of evaluation will provide performance feedback

and permit periodic assessment of progress toward achieving intended outcomes.

Results from the evaluation will be used to regularly monitor the project’s progress

toward meeting its objectives and intended outcomes, and will prompt specific recommendations

for program improvement. Formative evaluation findings also will be used to assess the extent

to which the project is ensuring relevance of program activities to participants’ needs and

adherence to the proposed design. Findings from both formative and summative evaluation

activities will be communicated to project staff on a consistent basis at Arts Achieve Steering

Committee meetings and informally through telephone and email communications.

Both formative and summative evaluation results will be more formally summarized and

presented as they are available in annual performance reports on the time schedule specified by

the USDOE and in local evaluation reports, which will be completed at the end of each school

year. In these reports, the evaluator will include a presentation of quantifiable, descriptive and

analytic findings, as well as a narrative explanation of the data and interpretation of findings.

Each evaluation report will explore obstacles encountered and strategies to overcome these

challenges, as well as detailed recommendations for future program improvements and

expansion.

new york city department of education (pdf) · 2015-10-07 · nyc doe arts achieve--proposal...

Documents