rochester arts impact study
DESCRIPTION
Rochester Arts Impact Study. Deborah Harloff, RCSD Executive Director of Visual and Performing Arts Andrew MacGowan , III Project Administrator and Arts Impact Study Evaluator . - PowerPoint PPT PresentationTRANSCRIPT
Rochester Arts Impact Study
Deborah Harloff, RCSDExecutive Director of Visual and Performing Arts
Andrew MacGowan , IIIProject Administrator and Arts Impact Study Evaluator
The research on arts integration is based on the hypotheses that there is a positive impact between integrated arts instruction and increased student achievement .
What is arts integration? Arts Integration is the process of combining
content of the arts with other classroom content such as English Language Arts, Science, Social Studies, or mathematics. This is done by making clear content connections between the various curricular areas. Visual art
DanceTheatre
Music
The Arts as an interventionLittle empirical dataStudent motivation
1st Time 20062006-2009Over 200 applicantsRanked 13th nationally of 25 awardeesPerfect score on evaluation design
“True Experimental Design”What and Why• “Medical model,” “True Experimental,”
“Randomized Trial” all mean the same thing• Until NCLB, very few (< dozen) randomized
trials in education• The most rigorous form of evaluation• “quasi-experimental” is the next best form of
evaluation
The RCSD Design9 randomly selected elementary schools (out
of 39)180 teachers4,000 students 7 grade levels (K-6)Remaining 29 schools served as the control
groupApproximately 30,000 students in all, over
the course of four years (baseline year + three years of treatment)
Statistically “robust”: results can be believed
MeasuresSixteen major New York and national
standardized tests were used, in English Language Arts, Mathematics, Science and Social Studies
Components of the grantTen session teaching artist residency Additional arts experience Supplies to implement the residency Ten hours of PD for teachers
Year One: SelectionComputer-generated randomization of
schoolsIntroduce project to building administratorsInterview and hire teaching artists Create planning documentGrade level planning sessions
4 Schools Opt OutBased on the following rational:
the school based planning team (SBPT) was concerned about the ten hours PD requirement
the teachers at one of the schools were concerned that they would be writing the curriculum for the teaching artist
another school cited the concern about accepting a new program when the school had a new principal.
Years Two & ThreeYear two one school decided to opt out of the
program During year three, some of the faculty at one
school refused to meet thereby essentially opting out of the program.
Due to scheduling conflicts and organizational skills one teaching artist was not retained.
A different teaching artist moved out of state and could not complete the project.
Year One: Implementation Grade level planning meetings were
scheduled
Schedules were arranged to accommodate teaching artist availability as well as classroom continuity.
Materials and supplies needed for the project were discussed.
Years Two & Three: Implementation Due to inconsistencies in planning meetings,
a planning document was developed to create uniform lesson and unit plans
Schedule protocol remained the same for year two.
Introduction of the 4 Art FormsThe art form that was integrated at the
school was randomly selected . Two schools were assigned as music
two were assigned as danceThree schools were assigned as theatre
Three were assigned as visual art
Skills and Content Taught Visual Art
MusicTheatre
Dance
Arts Impact Study PDProfessional development (PD) to reinforce
understanding of integration techniques and strategies in music, dance, theater and visual arts.
Professional reading 96% approval ratingHands onModels strategiesProvides materials
PD for Teaching Artists Program objectives, program implementation and
classroom residencies. Operational practices were taught to the teaching
artistsTechniques to reinforce ELA, Math, Science and
Social Studies with the teaching artists’ area of expertise.
Collegial sharing - discussed best practices and successful strategies for co-teaching and co-planning lessons with classroom teachers.
Main ConclusionsUsing both national and NY tests, positive and
significant effects, from 5% to 30% improvements
Across most grade levels K – 6Students with Disabilities showed gainsEnglish Language Learners showed gainsGains in both low and high performing schoolsGains in ELA and MathScience and Social Studies (not statistically
significant)
More conclusionsThe lower the grade level, the greater the effects
(e.g., the younger the student, the greater the effects);
The more economically disadvantaged the student, the greater the effects.
Major, unexpected finding: Arts Impact acts as a protective factor for students who did not have the benefit of attending a higher performing school.
Top conclusions (charts to follow)Kindergarten through 2nd grade students in the
treatment schools enjoyed the greatest degree of gains, including Students With Disabilities.
By Year 3, Kindergarten students realized a 14% differential, 56% of the treatment students reached Hi/Scope Child Observation Record (COR) Proficiency Level 4 (on the COR 5-point scale), versus 41% of the control students.
The gains in Year 3 for Kindergarteners With a Disability were even greater, with 25% more gaining on the COR Proficiency Level 4 (on a 5-point scale), 30% for the control students vs. 55% for treatment students.
The comparative gains for Kindergarten English Language Learners in Year 2 saw an approximate 4% advantage.
By Year 3, 1st grade students on Free or Reduced-Price Lunch (an indication of poverty), realized a 6% comparative gain in Reading, Language Arts and Mathematics (identical in all three) as measured by the CTB Terra Nova (which is configured to New York State Standards.)
More conclusions . . .By Year 3, 1st grade English Language Learners realized a 12% comparative
gain in Reading, an 18% comparative gain in Language Arts and a 13% comparative gain in Mathematics.
By Year 3, 1st grade Students With Disabilities realized a 15% comparative gain in Reading, a 6% gain in Language Arts and an 8% gain in Mathematics, compared to control school students.
For reasons we do not yet understand the intermediate grades (3 – 6) see greater gains for students in 4th and 6th grade than in 3rd and 5th grade.
In low performing schools, 4th grade students realized a 13% higher rate of English Language proficiency and a 10% higher rate of Mathematics proficiency (as shown by the New York State Tests), compared to low performing control schools.
In low performing schools, 6th grade student gains were noteworthy -- a 13% higher rate of treatment students achieving English Language Arts proficiency (New York State Levels 3 and 4), than control school students, and a 20% higher rate of treatment students achieved Mathematics proficiency compared to students in the control schools.
Gains in Kindergarten – all studentsVII: Rochester Kindergarten Students
Improvement in English Language Arts, Math, Science and Behavior
0
10
20
30
40
50
60
70
80
90
Fall, 2006 Spring, 2007 Fall, 2007 Spring, 2008(p<.01)
Fall, 2008 Spring, 2009(p<.001)
Perc
ent S
corin
g at
or a
bove
4 o
n th
e To
tal K
-CO
R 5
leve
l Sca
le
Control SchoolsProgram Schools
(p<.001)
Martin F. Gardiner, Ph.D.
Gains for Grade 1 ELLV: Rochester 1st Grade Students With Disabilities
Progress in English Language Arts and Math
0
5
10
15
20
25
30
35
40
45
Reading2007-2008
(p< .05)
Reading2008-2009
NYLanguage2007-2008
NYLanguage
2008-2009
NY Math2007-2008
(p < .01)
NY Math2008-2009
Perc
ent S
corin
g at
or a
bove
Gra
de L
evel
Control Schools Program Schools
Year 1
Reading (2006-2007)
NY Lang.(2006-2007)
(p < .01)
NY Math(2006-2007)
Martin F. Gardiner, Ph.D.
Gains for Grade 1 Students with Disabilities
V: Rochester 1st Grade Students With DisabilitiesProgress in English Language Arts and Math
0
5
10
15
20
25
30
35
40
45
Reading2007-2008
(p< .05)
Reading2008-2009
NYLanguage2007-2008
NYLanguage
2008-2009
NY Math2007-2008
(p < .01)
NY Math2008-2009
Perc
ent S
corin
g at
or a
bove
Gra
de L
evel
Control Schools Program Schools
Year 1
Reading (2006-2007)
NY Lang.(2006-2007)
(p < .01)
NY Math(2006-2007)
Martin F. Gardiner, Ph.D.
Quantitative Data143 teacher interviews were conducted at the
end of the third year of the grant project1. What worked best?
the arts activity 30 21%supported content goals 26 18%the integration and the artist sessions
21 15%
student performance 12 8%movement activities 5 3%hands-on aspect of the program 5 3%teaching artist was organized 5 3%Particular teaching artist 5 3%flexibility of the teaching artist 4 2.7%class participation 3 2%artist modeling for the students 3 2%PD session 2 Culturally based activities 1 Planning together 1 Picture books 1 Illustrating 1 Engage students 1 Team work activities 1
2. What is your evidence of student learning?Response # of
responsesPercentage
Student performance or project produced
31 22%
Students engagement increased 25 17%Application of knowledge from lesson to classroom discussion
14 10%
Student recall 40 28%Use of vocabulary 10 7%Students transfer of knowledge form one content area to another
2
Test scores 2 Student writing 1 Student confidence increased 1
3. What activities that the teaching artist modeled have you tried or would you do?
Music activity 36 25%Movement/ dance activity 34 24%Theatre activity 34 24%Visual art activity 31 22%Changing environment – took kids to another space
2
Cooperative activities 4 3%Make costumes 1 Make musical instruments 3
4. What support do you need to continue with arts integration?
Responses # of Responses
Percentage
Materials, equipment, resources 45 31%Professional development 40 28%Funding 7 5%More time during the day ( time constraints)
18 13%
More Teaching Artist sessions 17 12%Support for using integrations 7 5%
Did it work?In short, Yes. 30,000 students and 16
standardized tests.
SummaryOverall, we have concluded the earlier the
intervention, the greater were the effects sizes: We observe statistically significant positive effects in Kindergarten, first and second grade. We also observed the higher the poverty as determined by the Free and Reduced Price Lunch code the greater the effect size
Evaluation TipsIf you are contracting with an independent
evaluation outfit, get your Non-Disclosure Agreement done ASAP;
Work with US DoE on this – and feel free to borrow from Rochester’s model (available upon request)
Go with unique IDs, so you can use your Lunch Code data – which can reveal big differences in effects size
NEVER, EVER release Lunch Code information with personally-identifiable data
What Next?The results of this project – and the
independent evaluation of this project – pose significant policy implications for high-poverty urban schools, including those with high populations of English Language Learners and Students with Disabilities.