provenzano_formative and outcome evaluation_el sistema music program part 1&2 - final

El Sistema Ann Arbor Music Program 1

Running head: EL SISTEMA ANN ARBOR MUSIC PROGRAM EVALUATION

Formative and Outcome Evaluation of

El Sistema Ann Arbor Music Program

Anthony M. Provenzano

University of Michigan

Evaluation Proposal Part I & II

[Logic Model Submitted Separately]


STATEMENT OF NEED

El Sistema is a renowned Venezuelan music intervention program that has inspired a

global movement due to its social impact on fighting poverty and improving the human

condition. The El Sistema model provides high quality music instruction to children from a

broad range of socio-economic backgrounds in over twenty-five countries around the world

(Fundacion, 2012). Internationally adopted this socially innovative paradigm has reached Asian,

European, and Anglo countries including the United States and Canada (Tunstall, 2012). Now

thriving in Los Angeles, Boston, Philadelphia, Toronto, and other major cities and music centers,

this famed program fills a special need in the corridor between Ann Arbor and Detroit. The

reasons for starting a pilot program in southeast Michigan are threefold: 1) serving socially,

racially, and linguistically diverse children in the University of Michigan (U-M) community;

2) planning the scale up of the El Sistema music intervention to include schools and other venues

in Detroit; and 3) developing a Social Engagement curriculum of greater depth for University of

Michigan School of Music, Theatre, and Dance (SMTD).

Two types of evaluations will be used to help shape the El Sistema music intervention in

Ann Arbor and to verify if the pilot program is effective in achieving its objectives. A formative

evaluation will be conducted to document the emerging curriculum from year one of the El

Sistema Ann Arbor pilot project. The results of this evaluation will be used to manualize the El

Sistema program for future replication and expansion purposes. An outcome evaluation will be

conducted to offer evidence that the El Sistema program is meeting its intended outcomes.

There are two spheres of influence that have been integral to guiding the implementation

of the El Sistema model, and designing the formative and outcome evaluation:

1) Ann Arbor Public Schools/Mitchell Elementary School; and

2) The University of Michigan School of Music, Theatre, and Dance.


Spheres of Influence / Aims of the Evaluation

Ann Arbor Public Schools/Mitchell Elementary School

Mitchell Elementary is a school of approximately 280 students in the southeast area of

Ann Arbor. The school population is diverse with about 30% African American, 40%

Hispanic/Latino, 5% Asian students, 22% Caucasian and more than 30% of the students

speaking a language other English at home. Approximately 80% of students qualify for free or

reduced lunch. Administrators from Ann Arbor Public Schools responsible for research and

evaluation, and the Principal and Music Teacher at Mitchell Elementary, have expressed the

need for improving school performance, student academic achievement, and relationships with

families and the community. In response, key stakeholders from the El Sistema initiative have

adapted the Venezuelan music model to develop a theory of change and logic model (Refer to

Appendix 1: Logic Model) to target the specific needs, goals, and objectives of Mitchell

Elementary and Ann Arbor Public Schools.

Mitchell Elementary School is in its third year of an intensive partnership with the

University Of Michigan School Of Education, the Mitchell Scarlett Teaching and Learning

Collaborative (MSTLC). This partnership is designed to assist school personnel to improve

school achievement and to create a context for teacher education. Through the El Sistema

initiative, Mitchell Elementary intends to build upon other after-school and extended year

programs offered by the MSTLC that focus on literacy, math, and science instruction. In

partnership with the University of Michigan SMTD, School of Social Work (SSW), and aligned

with the El Sistema philosophy, Mitchell Elementary has established the following school-

specific program objectives for the El Sistema pilot program: 1) improving student academic

performance; 2) cultivating a strong sense of school pride; and 3) increasing family and


community engagement. The purpose of the evaluation is to document school and student

outcomes related to music literacy, individual and collective empowerment, and sense of

community and school pride; and to collect formative research data during the course of the

outcome evaluation to shape the El Sistema program.

University of Michigan School of Music, Theatre, and Dance

The University of Michigan School of Music, Theatre, and Dance aims to develop a

social engagement pedagogy and community-focused curriculum for SMTD students through

their participation in the El Sistema initiative. This objective is central to SMTD’s mission and

development as a professional performing arts school. Teaching students how to engage

communities in which they work is now a core skill for the 21st Century performing artist and

arts educator. A social engagement curriculum situates SMTD students in the center of the global

concerns surrounding income inequality, economic deprivation, lack of access to education,

health care and other basic needs. It allows students opportunities to witness how their artistry

speaks to and transforms communities coping with these stressors. Through the collaborative

partnership with Mitchell Elementary School the El Sistema model will be used as the catalyst to

form a social engagement pedagogy and curriculum. By enriching SMTD’s curriculum with

social engagement teaching activities graduate students will: 1) develop a sense of social

consciousness; 2) increase opportunities for graduate students to provide instruction across

different community settings; and 3) increase employability through traditional and non-

traditional curricula. The purpose of the evaluation is to document the teaching experiences of

SMTD students, their development of a sense of social consciousness, and their perspective on

the potential for increased employment opportunities. The evaluation also aims to collect


formative research data to document the pedagogical approach used by the El Sistema model to

help form a social engagement curriculum for SMTD.

Key Stakeholders

El Sistema Ann Arbor is partially funded by The University of Michigan Third Century

Initiative, a collaboration between the School of Music, Theatre, and Dance and the School of

Social Work. The goal of the Third Century Initiative is to support interdisciplinary research

aimed at discovering innovative pedagogies that provide U-M students unique learning

experiences in the community. In this engagement, SMTD is responsible for implementing a

pilot program at Mitchell Elementary School using the El Sistema model. SMTD graduate

students with previous involvement in El Sistema provide music instruction in partnership with

Mitchell Elementary School’s Music Teacher. In addition to teaching children to play music as

part of the El Sistema program, the SMTD graduate students are also key informants in the

evaluation, helping to guide the development of SMTD’s social engagement pedagogy. Fifth

grade string instrument students from Mitchell Elementary School are eligible to enroll in the El

Sistema after-school program and will also be assessed as part of outcome evaluation.

Graduate students from the School of Social Work Curtis Center Program Evaluation

Group (CC-PEG) are tasked with collectively designing and conducting the evaluation of El

Sistema Ann Arbor under the direction of the Associate Dean and CC-PEG Director. In the third

year of the Mitchell Scarlett Teaching and Learning Collaborative, the School of Education is

also a partner in the El Sistema initiative, providing input and feedback on the implementation

and evaluation. Key leadership from Ann Arbor Public Schools who are responsible for

elementary education and research and evaluation provide direction and support in developing

the pilot program and evaluation plan.


EVALUATION QUESTIONS

Outcome Evaluation

1. What impact does participation in El Sistema Ann Arbor have on fundamentals of string instrument

technique (posture, instrument position, bow hold, bow stroke, tone quality, intonation), music

performance skills (performing rehearsed music, sight-reading music, playing from memory, playing

by ear, improvising), and selected aspects of music literacy?

2. What impact does El Sistema Ann Arbor have on students’ musical self-esteem, academic

achievement, and conduct? 3. Have students increasingly acknowledged their sense of control in evoking change in their own lives? 4. Have students become more aware of their community?

5. Has their sense of belonging or contribution to the community increased? 6. Do students who participate in El Sistema Ann Arbor have higher levels of enrollment in the middle

school instrument program? 7. Has Mitchell Elementary School developed a stronger sense of pride as a result of implementing the

El Sistema program? 8. Has the School developed stronger connections with students, families, and the community?

9. Have the SMTD graduate students developed a stronger sense of social consciousness through their

work at Mitchell Elementary using the El Sistema model?

Formative Evaluation

1. What is the curricular content of the El Sistema model?

2. What is occurring during music lessons to create the music culture of El Sistema?

3. What are the similarities and differences in the curricular content from other forms of group class

string instruction typically found in the United States? 4. What are the skills the SMTD Instructors and Mitchell Elementary Music Teacher possess?

5. What are the concepts and key elements of the El Sistema pedagogical approach?

6. How can the El Sistema model best inform a social engagement curriculum?


INTERVENTION: EL SISTEMA ANN ARBOR PILOT PROJECT

The philosophical underpinnings of the El Sistema intervention focus on personal

development as a means to build community. “Music nourishes and is nourished by that day-to-

day pace, awakening its aesthetical sense, encouraging without artificial postures the emergence

of harmony and beauty in places such as the city, the country and even within the very

individual (Fundacion, 2013).” The El Sistema model supports children and youth through the

philosophy that music can provide “self-concept, self-esteem, self-confidence, discipline,

patience, and commitment.” This helps young people to recognize how their individual actions

impact collective goals, while cultivating a supportive environment of families, schools, and

communities. At the core of El Sistema’s community engagement approach is the concept that

music provides a “spiritual” richness that awakens the human condition through a heightened

state of mind, greater awareness of ethical principles, and improved intellectual and emotional

skills conducive to overcoming poverty (Fundacion, 2013).

El Sistema Ann Arbor has developed a theory of change to incorporate the guiding

philosophical principles of the El Sistema literature, a review of social and music learning

theories, and the specific community needs of Ann Arbor. Precepts include music as an agent of

change, excellence, community, and supporting personal growth. The theory of change states:

“If children participate in regularly and extensive in high-quality,

ensemble-based music education, they will develop a strong sense

of personal agency, and sense of community through developing

musical literacy (Landin, A,, et al 2013.)”

El Sistema Ann Arbor Theory of Change


Target Populations

There are two populations that will directly benefit from the El Sistema intervention.

Students from Mitchell Elementary School will not only receive high-quality music education,

but they will also benefit from individual growth, improved academic performance, and a

heightened sense of connection to their school and community. SMTD graduate students will

benefit from their experience teaching diverse communities, obtaining experience working with

schools, and recognizing the intrinsic value of working in non-traditional settings.

Conceptual Framework

The theory of change and logic model provides an overview of how the El Sistema Ann

Arbor intervention will achieve its intended outcomes (Refer to Appendix 1: Logic Model). A

key underlining assumption of the program intervention is that ensemble-based music instruction

will improve students’ music literacy, heighten a sense of personal agency, and further develop

their sense of community. Resources for the project intervention include students, school,

musical instruments, music teacher, SMTD graduate student instructors, transportation, and

afternoon snacks.

Goal 1: Offer intensive after-school group music instruction over the course of 26 weeks for

fifth grade string instrument students at Mitchell Elementary School.

Objective 1.1: Increase the students’ ability to read/comprehend music, establish collective

music making skills, and develop an interest in music.

Objective 1.2: Increase students’ awareness of individual and collective responsibilities, self-

care, and respect for others.


Objective 1.3: Increase students’ ability to build new relationships, experience community

through ensemble-making music making, and develop a sense of school pride

in relation to the music program.

Other assumptions posited by the logic model include the expectations of Mitchell

Elementary School and the University of Michigan School of Music, Theatre, and Dance. The

designed intervention aims to cultivate a strong sense of school pride and will be assessed by

measuring Mitchell Elementary School’s relationships with students, families, and the

community, the School’s connection to the El Sistema movement, and the potential for

generating additional resources for the School as a result of the intervention. The resources

required to support this objective include the University of Michigan’s performing arts theatre,

music performers from the community, principal, parents, music instructors, performance

audiences, and transportation.

Goal 2: Host guest musical performances at Mitchell Elementary, provide transportation

for students to attend musical performances in community, and offer transportation for

parents to attend their child’s music concerts.

Objective 2.1: Increase school pride among student, teachers, families, and administrators

through activities of the El Sistema program.

Objective 2.2: Increase the school’s connection with students, families, and the community

through the activities of the El Sistema program.

Objective 2.3: Establish a school connection to a national movement through El Sistema.

Objective 2.4: Capture the media buzz surrounding the activities of the El Sistema initiative.


The University of Michigan School of Music, Theatre, and Dance aims to develop a

social engagement curriculum by better understanding the El Sistema pedagogical approach, and

through the support of SMTD students with increased opportunities for teaching in non-

traditional community settings. Resources from the project that contribute to achieving this

objective include U-M Third Century Initiative funding, U-M School of Social Work, School

Education, Ann Arbor Public Schools, Mitchell Elementary School, SMDT graduate instructors,

and the community of Ann Arbor.

Goal 3: Use the El Sistema pedagogical model to offer SMTD students untraditional

community teaching opportunities for the purpose of developing a social engagement

curriculum.

Objective 3.1: Increase student teaching opportunities to include diverse populations.

Objective 3.2: Increase for SMTD students’ sense of social consciousness.

Objective 3.3: Manualize the El Sistema Ann Arbor program and develop social engagement

training modules to enhance SMTD’s curriculum.

Intervention Strategies

v To maximize program recruitment and retention, students identified to participate in the

El Sistema program were already receiving instrumental music instruction as a standard

component of Mitchell Elementary School’s fifth grade music curriculum.

v In an effort to ensure student and parent participation in El Sistema Ann Arbor program,

students will be provided transportation to all activities including daily music instruction,

civic engagements, and musical performances. Parents will be provided transportation to

all five of their student’s musical performances.


v The SMTD music instructors have participated in El Sistema global movement as

students, and later as instructors. The previous experience of the SMTD graduate

students is an essential training component that directly impacts the success of El Sistema

intervention at Mitchell Elementary School and development of SMTD’s pedagogy.

v Key stakeholders for the El Sistema Ann Arbor Initiative are using a community-based

participatory approach to design the program intervention and evaluation plan.

REVIEW OF THE LITERATURE

El Sistema is a global movement that began thirty-eight years ago by Jose Antonio

Abreu, an economist and musician, discouraged by the fact that Venezuelans could not

participate in classical music. History tells us Abreu announced his plans to start a youth

orchestra in the burgeoning cosmopolitan capital city of Caracas, Venezuela. The initial

rehearsal took place in an abandoned parking structure with only eleven student musicians

showing up (Tunstall, 2012). As the orchestra grew, so did Abreu’s understanding of what was

being created – not only a music center, but a catalyst for changing social and emotional lives of

children and their families (Tunstall, 2012, p. 71).

“To me, an orchestra is first and foremost a way to encourage better

human development within children. That is why I always said, and

I say today, that [El Sistema] is not an artistic program but a human

development program through music. It is very important to be clear

about this. Because everything that happened then, and everything that

happened since then, has been a direct consequence of this concept.”

Jose Antonio Abreu


Today, there are over 300 youth orchestras or “nucleos” throughout the Venezuela with

the majority funding provided by their federal government. Approximately 370,000 children and

youth currently participate in El Sistema, in country that has population of approximately 28

million people. Of the children and youth in the El Sistema program, there have been estimates

that between 70 and 90 percent live in poverty (Tunstall, 2012). Mitchell Elementary School has

a similar rate of poverty, approximately 80 percent, which is measured by the proportion of

students receiving free or reduced lunch. Modeling the Venezuelan program, El Sistema Ann

Arbor will deliver free music instruction, provide instruments, transportation, uniforms,

nutritious snacks, and offer access to social services.

Fundacion Musical Simon Bolivar is the Venezuelan government foundation that

administers the El Sistema program, and in 2010 had an operating budget of $120 million.

Reports from the Intra-American Development Bank (IDB), a private funder of Fundacion

Musical Simon Bolivar provides evidence that El Sistema participants have demonstrated better

academic achievement and fewer behavioral problems than children and youth who are not

involved with El Sistema (IDB, 2013). Furthermore, Venezuela’s national high school drop-out

rate is over 26 percent, although only 6.9 percent for students who have participated in the El

Sistema program (Tunstall, 2012).

In the United States, El Sistema Colorado has experienced similar success with half of

teachers at Garden Place Elementary reporting positive changes in the academic performance of

students who participated in El Sistema. These results did not show a reduction in drop-out

rates. However, it did highlight strong performance indicators that support the school’s goals in

achieving long-term retention and success in school. Data from El Sistema Colorado also

demonstrated higher levels of reading proficiency (approximately 9% greater) for students in the


El Sistema program over non-El Sistema students. Results also depicted improvements to self-

discipline (with an average increase of 8.9%), and self-control (averaging an increase of 12.5%)

among students in the El Sistema program (El Sistema Colorado, 2013). El Sistema Colorado

offers evidence of the impact the El Sistema model has on student academic achievement and

behavior in the United States. Other research studies have provided evidence that students who

engage in learning to play a music instrument had significantly higher self-efficacy scores than

those who didn’t (Ritchie, L., & Williamon, A., 2011). In one particular study, researchers

provided evidence of strong positive correlation between self-efficacy and child well-being

(Ritchie, L., & Williamon, A., 2011).

In an attempt to replicate similar findings, El Sistema Ann Arbor will evaluate students’

sense of personal agency (self-efficacy, personal responsibility, self-care, and respect of others),

sense of community, sense of well-being, and perceived self-esteem in relation to their musical

ability. The evaluation also will examine student academic achievement and behavioral conduct

through a review of official school records that capture student attendance, grades, and behavior

to show correlations between increased sense of personal agency and academic achievement.

The replication of El Sistema in United States looks significantly different than

Venezuela’s centralized government funded system. In the United States El Sistema programs

are organically grown out of music centers, philanthropic organizations, non-profits, and

universities, and challenged with the attempt to sustain funding and scale up efforts.

Furthermore, in the United States public and private funders and researchers are more concerned

with creating quantifiable measures for the purpose of evaluating the effectiveness of the of El

Sistema intervention (Tunstall, 2012).


Although there is incomplete data on the effectiveness of the El Sistema model and

related outcomes, El Sistema Ann Arbor has incorporated a variety of psychological measures

supported by social and music learning theories. Furthermore, there is limited literature on the

El Sistema pedagogical approach. However, the El Sistema philosophy is adequately

documented. In fact, El Sistema Ann Arbor has ascribed to these ten fundamental principles,

Refer to Appendix 2: El Sistema Ann Arbor Fundamental Principles for the details of each

respective component:

1. Mission of social change.

2. Access and excellence.

3. The nucleo environment.

4. Intensity.

5. The use of ensemble.

6. The CATS teacher model: Citizen/Artist/Teacher/Scholar.

7. The multi-year continuum.

8. Family and community inclusion.

9. Connections and network.

10. Ambition and Achievement.

Eric Booth, El Sistema USA, 2010 CONCLUSION

A literature review of social and music learning theories was conducted to reveal

strategies to complement the limited data on the El Sistema pedagogical approach and evaluation

research. The racial, ethnic, linguistic, and socio-economic diversity, added to limited resources

of the Mitchell Elementary community, affirms the selection of the target population and

highlights the need for a socially innovative intervention to improve overall school performance

and the development of SMTD’s social engagement pedagogy.


RESEARCH DESIGN

A single group pretest-posttest multi-method research design will be used to evaluate the

effectiveness of the El Sistema Ann Arbor pilot project in achieving its desired program goals,

objectives, and outcomes, as determined by the proposed conceptual framework and logic model

(Refer to Appendix 1: El Sistema Ann Arbor Logic Model).

Single Group Pretest-Posttest Design

O1 X O2

X = Intervention O = Observation

The El Sistema Collaborative Partnership selected a pre-experimental design to deliver a cost

effective descriptive account of the Venezuelan model. Short time constraints leading up to the

project kick-off, and limited financial resources to support the implementation and evaluation,

were cited by the Collaborative as reasons for selecting this type of design. Although the pre-

experimental design is feasible provided the restraints placed on the community, this study

design presents limitations, and positive outcomes could be a result of factors other than the

program intervention.

Threats of validity and reliability exist because this design lacks scientific rigor. The

small sample size threatens the study’s reliability, and threats to validity are evident, particularly

maturation, given the intervention is delivered to elementary school-aged children. However,

when feasible, testing the impact, attrition, and instrumental effect will be monitored and

managed through the implementation of the evaluation protocols. Despite these limitations, this

type of design will provide key stakeholders with information to better understand the target

population, services, and attainment of expected outcomes for future replication and expansion

purposes.


Study Recruitment and Retention

A purposeful sample will be used to maximize program recruitment and retention, and to

help control project costs. Twenty-two students at Mitchell Elementary School who currently

receive string instrumental instruction as a standard component of the 5th grade music curriculum

in Ann Arbor Public Schools will be targeted for this study. These students have been selected

to participate in the program because the El Sistema trained SMTD graduate instructors are an

experienced violinist/violist and cellist. Naturally this has excluded fifth grade horn instrument

students from participating in the after school program.

The parents of string students will be mailed letters providing information regarding the

El Sistema after school program and inviting them attend a program orientation. All

communication to the parents regarding the after school program will be provided in both

English and Spanish. During the orientation family members will be provided with: 1) the El

Sistema program permission slip for student enrollment; 2) the parental consent form for their

child’s participation in the program evaluation; and 3) a one item questionnaire regarding the

expectations for their child in the after school program. In an effort to increase program

participation and ensure retention, project staff will contact families via the telephone to inquire

about reasons for not participating, and to discover potential causes for students dropping-out.

Parental consent forms will be translated for Spanish speaking families, and bilingual

project staff will be available during the program orientation to answer any questions parents

have regarding student participation in the program and the exploratory study. Student

participation in the study is voluntary. A student’s experience in the El Sistema program will not

be positively or negatively impacted as a result of their participation in the evaluation. Students

have the right to refuse to answer any of questions, and at any point can choose not to participate


in the study. No one will be able to identify students from any of the data, analysis, findings, or

reports. Signed parental consents forms and sensitive survey data will be kept in a locked office

and located on the University of Michigan’s secured password protected computer network.

Data Collection and Measurements

The multi-method data collection processes for the program evaluation will be reviewed and

approved by the University of Michigan Institutional Review Board. The quantitative

procedures for this study design consist of conducting pretest (O1) and posttest (O2) student

surveys and teacher ratings. The psychosocial surveys and teacher ratings have been researched

and collectively identified to systematically align the selected measures with the objectives of the

El Sistema music intervention. Quantitative data collected from five student surveys and one

teacher rating survey will be triangulated with qualitative data collected from parents and music

instructors to add depth to the analysis. Students and 5th grade teachers will be evaluated using

the following proven and reliable measures:

KEY INFORMANT

NAME OF MEASURE AUTHOR(S) DATE DEMONSTRATED EFFECTIVE

IN MEASURING

Students Attitudes Towards School

Anderson, S. A. 1999 Student’s perception of school

Students Satisfaction With Life

Gadermann, A. M., et al 2010 Individual well-being

Students Conflict Behavior Kammratrh, L. & Dweck, C. S. 2006 Relationship Conflict Behavior

Students Parent Involvement In Students’ Music Education

Zdinski, S. 1993 Music environment at home

Students Self-Esteem of Music Ability

Schmitt, M. & Zdinski, S. 1979

Self-confidence; Skills and abilities in music; and Feelings of acceptance and reinforcement by parents, teachers, and friends

5th Grade Teachers

School Achievement Motivation Chiu, L. H. 1997

Teacher rating of student achievement motivation demonstrated in the classroom


The qualitative procedures include administering semi-structured interviews, focus

groups, and questionnaires to study participants at time O1 and O2. Interview and focus group

questions for music instructors, school faculty, and parents will address student outcomes in

relation to music literacy, sense of community, and sense of personal agency; perceived sense of

school pride; and the music instructors’ perceived sense of social consciousness. Findings from

the outcome evaluation will demonstrate the effectiveness of the program.

Digital video recordings of a sample of music instruction lessons will conducted to assess

instrumental technique and other aspects of music literacy. The evaluation also will examine

student academic achievement and behavioral conduct through a review of official school

records that capture student attendance, grades, and behavior to show correlations between

increased sense of personal agency and academic achievement.

Analysis Plan

Upon completion of collecting pretest and posttest data, SPSS software Version 21.0 will

be used to organize, manage, and analyze data to evaluate the effectiveness of the intervention

outcomes. A paired analysis of individual student surveys will be conducted. A data dictionary

will be developed to help organize and facilitate the data entry processes. Descriptive statistics

will be performed to report frequencies. Inferential statistics will be computed to explore

possible associations between measures. Chi-square tests of independence will be conducted to

identify significance at p<.05. Associations among student characteristics, school grades, dosage

(attendance), and outcomes will be analyzed. The table below provides the quantitative measure,

level of measurement, and statistical technique corresponding to the specific program objectives.

For a list of the proposal objectives, refer to pages 8 – 10.


QUANTIATIVE ANALYSIS PLAN

Objective(s) Measure Level of Measurement Statistical Technique

1.1 Student Grades Ordinal Chi-Squared Test and Percentages

1.1 1.2 Students’ self-esteem of music ability Ordinal Chi-Squared Test and

Percentages

1.2 Students Behavior – Number of suspensions Ratio Mean/Standard Deviation

1.2 1.3 Students’ conflict behavior Ordinal Chi-Squared Test and

Percentages 1.2 1.3

Teacher ratings of student school achievement motivation Ordinal Chi-Squared Test and

Percentages

1.3 Students’ attitudes towards school Ordinal Chi-Squared Test and Percentages

1.3 Students’ satisfaction with life Ordinal Chi-Squared Test and Percentages

1.3 Students’ perception of their parent involvement in their music education Ordinal Chi-Squared Test and

Percentages

N/A

Students’ age in years Interval Mean/Standard Deviation

Students’ gender Nominal Frequencies and Percentages

Students’ Race/Ethnicity Nominal Frequencies and Percentages

Number of students attending each music lesson Ratio Mean/Standard Deviation

Semi-structured interviews with graduate student music instructors, elementary music

school teacher, and school principal will be audio recorded and transcribed by the evaluation

team. The qualitative data collection will also include collecting parent questionnaires and

conducting a posttest focus group with family members. The focus group will be audio-recorded

and transcribed. Graduate students from the School of Social Work will analyze transcripts and

open-ended responses on questionnaires looking for concepts and themes. The qualitative data

collected will be organized and coded into categories using NVivo software. The table below

provides the qualitative measure corresponding to the specific program objectives in this

proposal. For a list of the proposal objectives, refer to pages 8 – 10.


QUALITATIVE ANALYSIS PLAN

Objective(s)

Measure

1.1 Students’ instrumental technique and the development of other aspects of music literacy

1.2 Students’ executive functioning skills (memory, planning, and managing time and attention)

1.2 1.3 Students’ sense of belonging

1.3 Students’ personal sense of control

2.1 2.2 Students’ and families’ connection to school

2.1 2.2 School faculty’s connection with families and community

2.2 Additional resources provided to the school

2.3 2.4 Community’s connection to a National movement

3.1 Experiences gained from graduate student instructors

3.2 Graduate students’ recognition of value added teaching in diverse community settings

3.3 Skills the graduate school instructors and music teacher possess.

3.3 Curricular content of an El Sistema style program

3.3 Similarities and differences in El Sistema model from other curricula

CONCLUSION

Inspired by El Sistema’s pedagogical approach, and its social impact on fighting poverty

and improving the human condition, this famed Venezuelan model has emerged as the catalyst

for the University of Michigan to develop an “innovative, multi-disciplinary teaching and

scholarship approach” (U-M, 2013). Partially funded by the University of Michigan’s Third

Century Initiative, the School of Music, Theatre and Dance (SMTD), School of Social Work, and

School of Education have partnered with Ann Arbor Public Schools to develop an innovative music

intervention program for diverse communities, and a social engagement curriculum of greater depth


for SMTD graduate students. Through this interdisciplinary research initiative, faculty, graduate

students, and key stakeholders will use a Community-Based Participatory Research approach to

collaborate on the design, implementation, and evaluation of the El Sistema Ann Arbor music

program.

Existing literature, although limited, provides evidence of the effects of the El Sistema model

on improving social, health, and education outcomes. Furthermore, there are few studies on the El

Sistema pedagogical approach, however, this is one area the proposed evaluation can make a

contribution to the field. Through this research partnership, formative and outcome evaluations

will be used to help shape the El Sistema music intervention program, enhance the SMTD social

engagement curriculum and pedagogy, and verify if the pilot program is effective in achieving its

objectives.

Transforming approaches to teaching and scholarship through multidisciplinary social

engagement opportunities are the precepts for this collaborative research partnership. By

drawing from the innovative pedagogical approach of El Sistema, and combing proven research

methods from music, social work, and education, faculty, graduate students, and community

partners will gain from the collective intelligence of the different disciplines. This shared

knowledge will provide opportunities for scholars and communities to develop advanced

philosophical approaches for responding to challenging health, social, education, and economic

concerns.


REFERENCES

Anderson, S. A. (1999). Attitudes toward school scale. Center for Applied Research, University

of Connecticut, School of Family Studies.

Booth, E. (2010). Fundamental of el sistema. El Sistema USA. Last accessed, November 4, 2013.

http://ericbooth.net/the-fundamentals-of-el-sistema/

Chiu, L. H. (1997) Development and validation of the school achievement motivation rating

scale. Educational and Psychological Measurement, 57, 292-305.

El Sistema Colorado. El sistema colorado accomplishments. Last accessed November 4, 2013.

http://www.elsistemacolorado.org/our-program/accomplishments/

Fundacion Musical Simon Bolivar, Homepage. Last accessed, November 1, 2013

http://www.fesnojiv.gob.ve/es/el-sistema.html

Gadermann, A. M., Schonert-Reichl, K. A., and Zumbo, B. D. (2010). Investigating validity

evidence of the satisfaction with life scale adapted for children. Social Indicators

Research, 96, (2), 229-247.

Inter-American Development Bank. Last accessed, November 2, 2013.

http://www.iadb.org/en/inter-american-development-bank,2837.html

Landin, A,, Roldan, C., Sandoval, E., and Zanussi, S. (2013). Say yes to assess: An exploration

in el sistema-inspired assessment practice. Sistema Fellows Program at New England

Conservatory.

Kammrath, L., & Dweck, C.S. (2006). Voicing conflict: Preferred conflict strategies among

incremental and entity theorists. Personality and Social Psychology Bulletin, 32, 1497–

1508.


Ritchie, L., and Williamon, A. (2011). Primary school children’s self-efficacy for music

learning. Journal of Research in Music Education, 59 (2), 146-161

Schmitt, M. (1979). Development and validation of a measure of self-esteem of music ability.

(Doctoral Dissertation, University of Illinois at Urbana-Champaign). Dissertation

Abstracts International, 40, 5357A. (University Microfilms No. 80-09164.

Tunstall, Tricia. (2012). Changing lives: gustavo dudamel, el sistema, and the transformative

power of music, New York, Norton.

University of Michigan. Third Century Initiative. Last accessed, December 16, 2013.

http://thirdcentury.umich.edu/about/

Zdzinski, S. (1994). Parental involvement, gender, and learning outcomes among

instrumentalists. Contributions to Music Education, 21, 73-89.


ABSTRACT

El Sistema is a renowned Venezuelan music intervention program that has inspired a

global movement due to its social impact on fighting poverty and improving the human

condition. The El Sistema model provides high-quality music instruction to children from a

broad range of socio-economic backgrounds in over twenty-five countries around the world

(Fundacion, 2012). Internationally adopted, this socially innovative paradigm fills a special need

in the corridor between Ann Arbor and Detroit, Michigan. This reasons for starting this pilot

program in southeast Michigan are threefold: 1) serving socially, racially, and linguistically

diverse children in the University of Michigan (U-M) community; 2) planning the scale up of the

El Sistema music intervention to include schools and other venues in Detroit; and 3) developing

a Social Engagement curriculum of greater depth for University of Michigan School of Music,

Theatre, and Dance (SMTD). A single group pretest-posttest multi-method research design will

be used to evaluate the effectiveness of the El Sistema Ann Arbor pilot project. The El Sistema

program will target 5th grade string instrument students at Mitchell Elementary School. If the

intervention is successful, it is expected that by having children participate in regularly and

extensively high-quality, ensemble music education students will develop music literacy, a

strong sense of personal agency, improved academic performance, and increased sense of

community. In addition, U-M graduate student music instructors will develop a sense of social

consciousness in delivering music instruction in untraditional community settings using the El

Sistema pedagogical model. Transforming approaches to teaching and scholarship through

multidisciplinary social engagement opportunities are the precepts for this collaborative research

partnership. By drawing from the innovative pedagogical approach of El Sistema, and combing

proven research methods from music, social work, and education, faculty, graduate students, and

community partners will gain from the collective intelligence of the different disciplines. This

shared knowledge will provide opportunities for scholars and communities to develop advanced

philosophical approaches for responding to challenging health, social, education, and economic

concerns.


Appendix 1: El Sistema Ann Arbor Logic Model

[Submitted As A Separate Document]


Appendix 2: El Sistema Ann Arbor Fundamental Principles

1. Mission of social change. El Sistema is a social change/youth development program that uses music to enable every child to feel like an asset within her or his community, inside and outside the “nucleo.” Students feel an ownership of the music making process, taking responsibility for both individual and group improvement.

2. Access and excellence. El Sistema includes as many children as it can, bringing young people into its community whenever possible, as young as possible, for as long as possible, whatever their background or abilities. As El Sistema strives single mindedly toward musical excellence for all students, it also provides intensive training at “Academies” for the most committed and gifted, preparing them for the highest-level national orchestras and cultivating them as leaders in their own communities. In this way and others, the ideals of access and excellence are maintained in a productive balance that maximizes both the fullest success for all and highest accomplishment for some.

3. The nucleo environment. The nucleo is a physical location, within the students’ neighborhood where students live, that embodies the values and goals of El Sistema. It is a haven of safety, fun, joy, and friendship, with an ethos of positivity and aspiration, where all students are encouraged to explore their potential. The nucleo‘s doors are always open, and community members convene in its hallways.

4. Intensity. Students spend a large amount of time at the nucleo, many hours per day, and almost all days of the week, often building up to four hours per day, six days per week. Rehearsals are fast paced and rigorous, demanding a durable commitment, personal responsibility, and a strong work ethic. Through frequent performances, students have many opportunities to excel and to share their accomplishments with their peers, family and community.

5. The use of ensemble. The learning in El Sistema is based in ensemble experience in which group achievement is balanced with individualized attention. The orchestra acts as a model society in which an atmosphere of competition between individuals is replaced by shared struggle. [Dr Abreu: “The orchestra is the only group that comes together with the sole purpose of agreement.”] Smaller ensembles and choruses adopt the same ethos.

6. The CATS teacher model: Citizen/Artist/Teacher/Scholar. Those who work at the nucleo take on many jobs and multiple roles in relationship to the students. By acting as citizens, artists, teachers and scholars, these adults encourage their students to develop holistically: as active musicians, helpful educators, inquisitive learners and responsible civic contributors.

7. The multi-year continuum. El Sistema provides a “conveyor belt” of services, supporting its students from early childhood into adulthood. Despite variation in resources and practices, all nucleos work toward a full program. The “Academies” and other national teams have formed lists of sequential repertoire, orchestral levels, and pedagogical practices that create a through line for every child’s learning. Although each nucleo is encouraged to develop programs that suit its community, shared practices and unified vision allow El Sistema to provide its students with a continuous musical experience. The learning process develops the ear as the fundamental tool before the visual.


8. Family and community inclusion. Family participation is an essential aspiration of El Sistema. Siblings often go to the same nucleo, parents attend classes with the youngest students, and families form the bulk of the audience at orchestra concerts. Many sites have parent musical ensembles, and all actively work to involve the community at large through outreach concerts.

9. Connections and network. Although nucleos run independently and customize their programs, they are strongly connected to the national leadership organization, which provides financial resources but more importantly gives the network a unified vision. Additionally, each nucleo is indispensably tied to the many other nucleos that form the El Sistema network. These interdependent relationships are manifested through events such as “seminarios,” which are intensive, project-based musical retreats where orchestras share repertoire, streamline technique, and build personal and institutional relationships. By uniting students and teachers from disparate parts of the country, the nucleo network embodies the El Sistema ideals of sharing and learning.

10. Ambition and Achievement. El Sistema work is more than merely good for young people. It aspires to transform youth lives, and widely succeeds in setting a healthier, fuller trajectory for stress young lives. What changes young lives is being part of an aligned committed community that reaches unreasonably high together, regularly, taking risks to achieve excellence that matters to them and to others. This cycle of ambitious yearning and achievement, repeated consistently for the sake of beauty and contribution to the community, changes, over time, thousands of small and larger choices a young person makes in creating his/her life.

Eric Booth, El Sistema USA, 2010


Appendix 3: El Sistema Parent Consent English Spanish Dual



Appendix 4: El Sistema Student Surveys


provenzano_formative and outcome evaluation_el sistema music program part 1&2 - final

Documents