developing a program and music teacher evaluation tool for music education - creating your own...
DESCRIPTION
Developing a Program and Music Teacher Evaluation Tool for Music Education - Creating Your Own Accountability Report. Douglas C. Orzolek University of St. Thomas St. Paul, Minnesota. Developing Evaluation Tools for Music Education. Session Outline External Factors Internal Factors - PowerPoint PPT PresentationTRANSCRIPT
Developing a Program and Music Teacher
Evaluation Tool for Music Education - Creating
Your Own Accountability Report
Douglas C. OrzolekUniversity of St. Thomas St. Paul,
Minnesota
Developing Evaluation Tools for Music Education
Session Outline External Factors Internal Factors Administrators/Supervisors Literature On Developing an Evaluation Tool The Teacher Performance Assessment An Accountability Report What Next?
Developing Evaluation Tools for Music Education
External Factors Attitude (Peck) * Need for Composite Rating / “% Problem” (Burling) * Inconsistency Policy from State to State, District to District VAMs (Darling-Hammond, 2012) “Transparency or Accuracy?” (Prince, Friedman) Evaluator training, understanding of music, criteria don’t
apply $ to develop a Proper Assessment Reliability and Validity Legal Issues and Policy Validity (Prince)
Developing Evaluation Tools for Music EducationEvaluation Components for
Teachers in Non-tested Subjects or Grades – D.C. IMPACT Program (Steele,
2010)
75% Administrator or Master Educator Observation
10% Student Growth on a Teacher-Chosen Measure
10% Commitment to the School Community
5% School-wide Value-Added
Developing Evaluation Tools for Music Education
Internal Factors How Our Work is Percieved (Orzolek) * Subjective/Affective (Maranzano, Barrett) * Student Achievement? Connected to Learning? (Bowman) One Size Does Not Fit All - Multifaceted Jobs – Create, Respond,
Perform; Classroom, Ensemble, Individual (Grant, Taebel) Criteria for Excellent Music Teaching? (Brophy, Taebel);
“Successful” Teaching versus “Good” Teaching (Barrett) More Research – Connect to Other Non-tested Areas? The “My Program” Problem – Evaluate me on Program Success “If we say that music kids are smart, then why are we afraid of
VAMs? Wouldn’t that prove our point?” (Sean Barker, UST ‘12)
Developing Evaluation Tools for Music Education
MSBA Survey: Accountability in Music (Orzolek, 2004)No Complaints
“Good” ConcertsSuccessful Trips
No Letters to the EditorTrophies
Athletic BandsKids Seem Happy
C&I Person is HappyA Local “Expert”
Mostly at Budget Time“What else is there?
Developing Evaluation Tools for Music Education
Developing Evaluation Tools for Music Education
Reminders from Music Leaders Literature Clear Goals and Objectives – Cowden and
Klotman (1991), 97 “Evaluation ought to involve looking critically at
the process of gathering and weighing evidence that will reveal changes in terms of a desirbable musical product.” – Weyland (1960), 295
Very Subjective – Snyder (1959), 114 Multifaceted – Dykema and Gehrkens (1941), 368 “Seek competent evaluation of your work.” –
Prescott (1938), 259
Developing Evaluation Tools for Music Education
On Developing Evaluation Tools – ExternalBurling (2012) (aka Pearson Corporation)
Define a construct – What is an effective educator? Deploy multiple indicators – What evidence characterizes good
teaching and school leaderhip? Develop a clear composite rating – What weights should each
indicator have and who should be involved in the decision? Clarify differentiated performance levels – What distinguishes
varying levels of educator effectiveness? Build strong data analysis and reporting tools – What does the
information reveal about student, educator, and school performance?
Improve Instructional Leadership and Practice – How can the information target professional development to boost educator practice, student learning outcomes, and school efficacy?
Developing Evaluation Tools for Music Education
On Developing Evaluation Tools – ExternalOthers
Promote consistency in any student measures used to assess teachers (Steele)
Implement tools after experimentation (Steele) Should supply teachers with feedback – “close the loop”
(Prince) Teachers should determine the extent to which
math/reading scores are included (Prince) Portfolio (Prince) Always use tools that are valid and reliable – Charlotte
Danielson. (Peck)
Developing Evaluation Tools for Music Education
On Developing Evaluation Tools - InternalBrophy (1993)
Personal characteristics, as evidenced through student-teacher, teacher-faculty, and teacher administration interactions;
Music competence and performance skill, as evidenced when teaching a lesson;
Effective use of nonverbal strategies, such as modeling/demonstrating;
Effective use of verbal strategies, such as eliciting performance from students;
Developing Evaluation Tools for Music Education
On Developing Evaluation Tools - InternalBrophy (1993)
Classroom management, including discipline, group participation, and the creation of a positive learning environment;
Effective planning for concept learning and aesthetic appreciation across a wide age span;
An objective assessment of teaching style based on empirically supported criteria;
Appropriate professional development activities should be recommended.
Developing Evaluation Tools for Music Education
Three Categories of Teacher EvaluationNAfME (2011)
Teacher Evaluation linked exclusively to student outcomes.
Teacher evaluations linked exclusively to teacher practice.
Teacher evaluations that combine measures of student outcomes and teacher practice.
Developing Evaluation Tools for Music Education
Question
How do we help prepare teachers for these evaluations?
Developing Evaluation Tools for Music Education
TPA Background Teacher Performance Assessment – Stanford,
AACTE, CCSSO, Pearson, MACTE. 21 states actively involved.
Assesses “Effectiveness” of Pre-service Educators
The entire assessment is outlined in a “TPA Handbook” – “The Performing Arts”.
Completed during Student Teaching Scored by Trained Scorers via E-folio
Developing Evaluation Tools for Music Education
How does the assessment work? “An Authentic Assessment of Teaching
Competence” Students complete 4 Tasks:
Describe their Plans for Instruction & Assessment; Engage Students in Learning; Assess Student Learning; Analyze Teaching.
Parameters “Central Focus of Learning Segment” 3-5 Lessons Evidence provided on Video Use of “Academic Language” Detailed Rubrics Provided
Developing Evaluation Tools for Music Education
Starting Point: A Voluntary
AccountabilityReport
2002 - 2003 Accountability ReportInstrumental Music Education Department - George Washington High School
Contact: Douglas C. Orzolek ([email protected])
Mission Statement : High Quality, Comprehensive Music Education
Goals/Results for 2003-2004 Increase number of national standards completed /
Implemented composition unit this year Improve student rankings in state portfolio assessments / See
below Enhance music theory understanding by adapting new
software / New software purchased and implemented Provide outstanding performance opportunities for students /
Performances were well received - CD attached!
National StandardsThis year we added a composition (Standard 4)unit to our instruction. Several guests spoke oftheir experiences as composers, we took a fieldtrip to the American Composer’s Forum andwrote our own pieces that were performedduring our recital at the end of the year.
State/District Standards11th grade portfolios were sent to ourneighboring schools for evaluation andassessment by music educators in thosedistricts. The following are the results basedupon established rubrics: 4 (Exemplary) - 15% (10% in 2002) 3 (Proficient) - 45% (40% in 2002) 2 (Developing) - 35% (45% in 2002) 1 (Poor/Incomplete) - 5% (5% in 2002)The results suggest that our students areperforming well in these assessments of their work. This year’s scores show animprovement from the previous year.
Comments from Professional Reviews“Strongest performing group I have heard in years.” - LeonardBernstein
Comments from Student/Peer Evaluations “I feel like I have a better understanding of my instrument
after I’ve studied a composer and her intentions.” “”Billy’s perceptions of the performance were much different
than mine. But, I appreciated his points - they reminded me that music can be something different for everyone.”
“Opera - I still don’t like it.”
Testing Results for 2003-2004 9th Graders completed the NAEP music skills written portion
and scored in the 72 percentile nationally. Music reading skills measured by the Watkins-Farnum
performance scale continue to improve. The mean etude completed by students in grades 9-12 was #6 in 2002 and rose to #8 in 2003.
Average SAT score of a student in band was 1425 (of 1600.)
Other Highlights 97% of our students participated in the MN solo/ensemble
festival this year. 35% received Superior and 40% received Excellent ratings.
Our Concert Band earned the top ranking of Superior at the large group festival.
Our Marching Band was selected to perform in the Rose Parade.
We commissioned a new work from composer Libby Larsen. 6 band members were selected to perform with the All-State
band. Our students gave 60 public performances during this past
year. 80% of senior band members are planning to attend a 4
year college.
Developing Evaluation Tools for Music Education
What Next? Top of NAfME Agenda – Policy Analysis, Leadership Stay Informed – Websites, Readings Share Stories – Forum, Meetings Coordinate and Advance Research Efforts - Grants The Reality - Multifaceted and Messy –
Experiment. This is about People not Composite Ratings. Until we agree on what constitutes effectiveness,
it will be difficult to measure and reward it.
Developing Evaluation Tools for Music Education
Darling-Hammond & Bransford (2005)Effective Teachers: Understand subject matter deeply and flexibly; Connect what is to be learned to students’ prior knowledge and
experience; Create effective scaffolds and supports for learning’ Use instructional strategies that help students draw connections, apply
what they’re learning, practice new skills, and monitor their own learning;
Assess student learning continuously and adapt teaching to student needs;
Provide clear standards, constant feedback, and opportunities for revising;
Develop and effectively manage a collaborative classroom in which all students have membership.
Developing Evaluation Tools for Music Education
“When a call for accountability elicits discussions about
education, the results produced are usually positive.”
R. Colwell, (2003) “The Status of Arts Assessment: Examples from Music”, Arts Education Policy Review,
105 (2), 16.