music student teachers’ perceptions of the edtpa’s effectiveness in assessing their teaching...
TRANSCRIPT
Music Student Teachers’ Perceptions of the edTPA’s Effectiveness in Assessing
their Teaching Abilities
David W. Snyder, Illinois State University
What is the edTPA?
• a model for student teacher assessment that follows the portfolio model put forth by the National Board of Professional Teaching Standards (NBPTS) and developed from a California-based assessment system called the Performance Assessment for California Teachers (Darling-Hammond, Newton, and Wei, 2012).
• The edTPA includes a review of a teacher candidate's teaching materials and short videos of the candidate teaching to document and demonstrate a candidate's ability to effectively teach his/her subject matter to all students (edtpa.aacte.org).
• student teacher candidates are assessed in the areas of planning, instruction and assessment using reflective writing assignments and video critiques of their own teaching. These performance–based portfolios are then assessed by outside evaluators who are themselves professional educators trained by Pearson. (edTPA K–12 Performing Arts Assessment Handbook, 2013).
Where are we on implementing the edTPA?•As of February 2014, there are 34 states who have either piloted it or are currently using it as an assessment
•Of those, six states (Hawaii, Washington, Minnesota, Wisconsin, Tennessee, and New York), currently require all their student teachers to take it
•Five other states (California, Illinois, Ohio, Massachusetts, and Georgia) will soon require or offer edTPA as an assessment option
•The remaining 23 states have at least one institution using the edTPA
Where are we on implementing the edTPA?
Where are we on implementing the edTPA?• Washington state required teacher candidates to take edTPA to complete teacher
preparation programs and receive licenses for most subject areas starting this January.
• In New York, starting in May 2014 teacher candidates applying for licenses will be
required to pass edTPA.
• Illinois will require the edTPA for teacher licensure starting the fall of 2015.
• Georgia will require edTPA for teacher licensure beginning in 2015-16.
• Wisconsin will require edTPA for teacher licensure starting in 2016-17. The assessment
will be used as one measure for teacher preparation program approval beginning in
2015-16.
• California is piloting the edTPA as a fourth option among already required teacher performance assessments.
• Minnesota's Board of Teaching, which approves institutions and licensure programs to
prepare teachers, now uses edTPA as one measure.
• Hawaii will require a performance-based assessment such as edTPA for teaching
candidates beginning in 2017.
• Tennessee now allows edTPA to be used as a substitute for a standardized test of teaching practice.
-Doug Orzolek, Univ. St. Thomas from USA Today , Oct 2013
How is the assessment scored?
• There are 3 tasks to be completed during the student teaching semester: Planning, Instruction and Assessment
• There are 5 rubrics within each task with a possible high score of 5 for each one
• The recommended range of scores for passing by SCALE (Stanford Center for Assessment, Learning and Equity) is 37-42.
How is the assessment scored?
PLANNING
Rubric1
Planning for Developing Student Knowledge
Rubric 2
Planning to Support Varied Student Learning Needs
Rubric 3
Using Knowledge of Students to Inform Teaching and Learning
Rubric 4
Identifying and Supporting Language Demands
Rubric 5
Planning Assessments to Monitor and Support Student Learning
How is the assessment scored?
INSTRUCTION
Rubric 6
Learning Environment/Rapport
Rubric 7
Engaging Students in Learning
Rubric 8
Deepening Student Learning
Rubric 9
Subject Specific Pedagogy/Modeling
Rubric 10
Analyzing Teaching Effectiveness
How is the assessment scored?
ASSESSMENT
Rubric 11
Analysis of Student Learning
Rubric 12
Providing Feedback to Guide Further Learning
Rubric 13
Student Use of Feedback
Rubric 14
Analyzing Students’ Language Use
Rubric 15
Using Assessment to Inform Instruction
What is required to pass?• SCALE (Stanford Center for Assessment, Learning and Equity) developed the edTPA and is recommending a range of scores from 37-42 for passing the edTPA.
• Different states are handling it differently:
• New York has set the score at 41 to pass
• Illinois will more than likely choose 39 or 40 as a passing score
• Washington is starting at 35 and raising it in increments each year
Elisa Palmer, Illinois State University
What is required to pass?Passing rates from second large-scale field test in 2013 by SCALE
Background on this current study
All student teachers from the fall semester of 2013 (N=15) at ISU participated
All students were assessed by official Pearson scorers AND completed a self-evaluation using the same rubrics
In addition, all subjects completed an exit survey soliciting opinions on the helpfulness of the edTPA in preparing them for teaching
Results from this study will be compared to the second large scale field test that was done in 2013 of teacher education candidates in fine arts completed by SCALE (N= 116)
How did we do?Passing rates from current study Fall 2013 (N=15)
Scores ranged from 38-52
Mean score 44.7
Cut score of 38 100% passed
Cut score of 39 94% passed
Cut score of 40 94% passed
How did we do?
2013 Fall student teachers Pearson results: K–12 Performing Arts (N=15)Lowest score in red, highest in greenTask 1: Planning Rubric 1 2 3 4 5Mean 3.20 3.36 3.00 2.93 3.06Std 0.41 0.48 0.68 0.45 0.17Task 2: Instruction Rubric 6 7 8 9 10Mean 3.16 2.90 3.00 2.86 2.80Std 0.36 0.28 0.68 0.54 0.41Task 3: AssessmentRubric 11 12 13 14 15Mean 2.90 3.20 2.36 2.70 3.23Std 0.47 0.79 0.61 0.62 0.70
How did they think they would do?
2013 Fall student teachers self-assessment: K–12 Performing Arts (N=15)Lowest score in red, highest in greenTask 1: Planning (agreed on lowest score)Rubric 1 2 3 4 5Mean 3.73 3.46 3.53 3.33 3.40Std 0.70 0.83 0.83 0.90 0.91Task 2: Instruction (agreed on highest score)Rubric 6 7 8 9 10Mean * 4.00 3.33 3.13 3.40 3.53Std 0.37 0.48 0.51 0.82 0.83Task 3: Assessment (did not agree)Rubric 11 12 13 14 15Mean * 3.60 3.46 3.06 2.93 3.33Std 0.47 0.79 0.61 0.62 0.70
How have others done?Performing Arts results from second large-scale field test in 2013 (SCALE)
2013 Field Test Content Areas: K–12 Performing Arts (N=116)Lowest score in red, highest in greenTask 1: Planning (agreed with self assessments)Rubric 1 2 3 4 5Mean 3.09 2.99 2.93 2.89 2.91Std 0.48 0.67 0.64 0.62 0.64Task 2: Instruction (agreed with Pearson assessment)Rubric 6 7 8 9 10Mean 2.95 2.74 2.76 2.89 2.46Std 0.69 0.60 0.67 0.52 0.64Task 3: Assessment (agreed on lowest score in Pearson assessment)Rubric 11 12 13 14 15Mean* 2.83 3.02 2.34 2.49 2.73Std 0.64 0.69 0.66 0.60 0.84
How have others done?
•Rubric 4 (Identifying and Supporting Language Demands)•Rubric 10 (Analyzing Teacher Effectiveness)•Rubric 13 (Student Use of Feedback)
Were the three lowest scoring rubrics in the Planning, Instruction and Assessment Tasks respectively in both the current study (N=15)and the national pilot study in 2013 (N=116)
How have others done?
•Rubric1 (Planning for Developing Student Knowledge) •Rubric 6 (Learning environment/Rapport)
Were the highest scoring rubrics in the Planning and Instruction Tasks in both the current study (N=15)and the pilot study in 2013 (N=116)
The Assessment Task rendered rubric 15 as the high score for the students in this study (N=15) and rubric 12 as the high score for the 2013 pilot study (N=116)
How have others done?
Doug Orzolek’s students at University of St. Thomas in Minnesota have been using this test for several years now and have the same three rubrics as their lowest average scores.
•Rubric 4 Identifying and Supporting Language Demands•Rubric 10 Analyzing Teacher Effectiveness•Rubric 13 Student Use of Feedback
So why the problems with these rubrics (4, 10, 13) ?The language demand portion of the assessment (rubric 4) is very unclear in the handbook and in scorer training…
•students are asked to pick one “language function” for all 3 lessons,•identify supporting vocabulary,•and identify an additional language demand, either syntax or discourse
So why the problems with these rubrics?•Language function should be the verb used in the learning objective, e.g., identify, analyze, compare, perform, create, etc.•The students being taught need to demonstrate their understanding of the vocabulary necessary for the lesson in some way•Student teachers have difficulty engaging in discourse
•“class discussion”…(student to student, teacher to student) focused on the main learning objective using proper vocabulary
•Nobody knows what syntax is•“conventions for organizing symbols”…this can include the governing principles of reading music, i.e., rests, notes, musical staff
So why the problems with these rubrics?• Rubric 10 requires student teachers to analyze their videos and propose changes to their instruction related specifically to student learning NOT to delivery and classroom management.•Rubric 13 requires student teachers to explain specifically how the feedback given to their students will be used to revise their current work
•Writing “more practice” will not result in a high score•Many student teachers don’t see their students again after completing the video sessions or they move on to a new piece or unit so they simply don’t address this in their commentary
What helped students prepare for the edTPA?
Which of the support measures provided by ISU was most helpful to you in completing the edTPA? Please rank in order from 1 (most helpful) to 5 (least helpful) or DNA (did not apply).
__Student teacher seminars
__Feedback from university supervisor
__Using the rubrics during a previous method class
X On-line edTPA handbook
__Deadlines provided during the semester
What did they think about the edTPA?
Which set of tasks in the edTPA process did you find most difficult to complete and follow? Please rank from 1 (least difficult) to 3 (most difficult).
X Planning
__Instruction
__Assessment
What did they think?Which set of tasks in the edTPA helped you improve your teaching the most?Please rank from 1 (most helpful) to 3 (least helpful).a. X Planning
b. __ Instruction
c. __ Assessment
What did they think?Which of the following commentaries/tasks under “Planning” was most difficult for you to complete and evaluate? Please rank from 1 (least difficult) to 5 (most difficult).
__Rubric 1 developing student knowledge
__Rubric 2 supporting student learning needs
__Rubric 3 knowledge of students informing teaching
X Rubric 4 language demands
__Rubric 5 planning assessment
This matches the actual results from both their self-assessment and the Pearson assessment
What did they think?Which of the following commentaries/tasks under “Instruction” was most difficult for you to complete and evaluate? Please rank from 1 (least difficult) to 5 (most difficult).
__Rubric 6 learning environment/rapport
__Rubric 7 engaging students in learning
X Rubric 8 deepening students learning
__Rubric 9 subject-specific pedagogy/modeling
__Rubric 10 analyzing teaching effectiveness
This DOES NOT match the actual results from the Pearson assessment
Rubric 8 requires student teachers to prompt their students to higher order thinking or application related to the learning objective (questioning.) Students in this study reported difficulty distinguishing the difference between rubric 7 and rubric 8.
What did they think?Which of the following commentaries/tasks under “Assessment” was most difficult for you to complete and evaluate? Please rank from 1 (least difficult) to 5 (most difficult).
__Rubric 11 analysis of student learning
__Rubric 12 providing feedback
__Rubric 13 student use of feedback
X Rubric 14 analyzing student’s language use
__Rubric 15 using assessment to inform instruction
This DOES NOT match the actual results from the Pearson assessment
Rubric 14 is the other language related rubric and has the same problems that rubric 4 does (misunderstanding of terms). Many students simply fail to assess the identified “language function” in their student assessments.
What did non-music students think?
According to Darling-Hammond and Hyler in 2013, “the vast majority of candidates and teacher educators feel this process (edTPA) strengthens their practice”.
What did these music students think?
Do you believe that participating in the edTPA process strengthened your practice as a music teacher?
1 (strongly agree) 2 (agree) 8 3 (disagree)64 (strongly disagree)1
Mean: 2.53
SD: .64
Questions?
David W. Snyder
This PowerPoint presentation can be found at: www.cfa.ilstu.edu/dsnyder/news