managing a paradigm shift in assessment: a matter or mindset
TRANSCRIPT
Managing a paradigm shift in assessment – a matter of
mindset
Isabela Villas Boas, Casa Thomas Jefferson, Brasilia, Brazil
Grade oriented
Private school ranking
Public school annual evaluation
Culture of teaching to the test
Fierce competition to get into a good university – national examination
Casa Thomas Jefferson at a glance
Founded in 196316,000 + studentsSix branches + school contracts + in-
companyAround 200 teachersChildren - adultsBasic - post-advancedCourse Supervision – standardizationBalanced assessment system, but
with a greater emphasis on summative tests
The “Flex” courses
Around 3,000 Ss
Only adults
Four ten-week modules a year
Typical adult students
Traditional assessment system
After ten weeks of instruction:
One day for written test: reading comprehension, grammar, vocabulary; about 60% of the test focused on grammar structures and functions
One day for oral test: done in pairs; students stand outside the classroom and wait for their turn (following the proficiency test model)
How it all started - E-teacher course
Assessment
Summative and Formative Practices in Language Learning and Teaching
Traditional assessment system
Written test + oral test at the end of ten-week module
Ten weeks of instruction: accumulation of content
Focus on grammar and written exercises
Student anxiety
More selected-response items on the test
Lack of alignment
Negative washback effect
Summative Assessment
Formative Assessment
Traditional Assessment
Alternative Assessment
Aims of the new assessment system
Very little feedback Actionable feedback
Poor alignment Better alignment
New assessment system
1 2 3 4 5 6
Short assessments - 10 to 20 minutes2 oral assessments, necessarilyAt least 1 reading, 1 listening, and 1 writingMostly performance assessment of grammar
100 points
Last day of class: Final assessment daySs take the assessments they missed during the module or that they would like to retake (to improve outcomes)
PILOT
Traditional tests administered at the end to compare scores; comparison of final grades with typical student performance
Constant student feedback
Constant teacher feedback
Formal survey with students at the end
Ö Students had similar scores on their assessments and on the summative test
Ö Students had similar or higher final grades than the final grades in previous modules
Ö Students felt more at ease with the pilot system; less pressure
Building a shared vision - E-teacher course
Assessment
Summative and Formative Practices in Language Learning and Teaching
Phase-in stage
ImplementationWorkshops on
assessment
Ongoing feedback on
the outcomes of the new
system
Class observations
Revision of assessment
tools
Feedback from teachers and students at
the end of the module
New systemOld system
Students
More communication in classContent is not accumulatedNo need to memorize rulesNeed to study more frequently = more effective learningLess stressfulMore accurate assessment of the student because it is ongoingThe oral assessments are less stressfulNo last-minute studying onlyLighter“Forces” students to come to classMakes me feel more comfortable in class
Students
Fewer assessments
A final test is more effective in measuring knowledge
Have the six assessments + final test
Too easy
Fully Satisfactorily Partiallly No
Understand reasons for change
89% 6% 0% 0%
Support change 61% 17% 6% 0%
Teachers
No accumulation of content to assess.
Allows for remedial work and teacher self-assessment.
Student progress is more visible.
Immediate and personalized feedback to students.
Encourages more teacher reflection.
Students are assessed at their best.
Less stressful for students.
Possibility to re-teach and re-assess.
Teachers
Too many assessments.
Students keep asking if there will be an assessment the next class.
A little more time-consuming for teachers.
Difficulty with the oral assessments.
Two pilot groups
Two modules of each course
Four modules of each course
All levels: all teachers
The assessment system is appropriate for the students’ profile.
Totally Most of the time
Partially No No response
Change in staff
Few but very loud complaintsToo many assessments; too much time spent on assessments. Why not have four or three at the most?
Assessments are too easy. Students get higher grades than they used to; they won’t be ready for the next level.
Sometimes a whole class is spent on the oral assessment.
Grammar is not being assessed.
Our mental models – theories about the way the world works – influence our actions.
Examine mental models: Unexamined mental models limit people’s ability to change.
Schools that Learn, Peter Senge
Expecting an assessment will capture excellence, mediocrity, and failure as an educational measure reinforces a mindset that some students will always fail. That mindset has no place in our schools.
How Can Schools Develop Assessments That Matter?By Jill Berkowicz and Ann Myers, March 22, 2016Education Week
Too many assessments; too much time spent on assessments. Why not have three or four at the most?
It is okay to spend two full class days on oral and written tests, but it doesn’t seem okay to spend this same amount of time on smaller doses of assessment.
Belief Mental Model
It’s too easy. Students get higher grades than they used to; they won’t be ready for the next level.
Assessment as a student ranking tool;
Testing drives learning;
Quality = difficulty.
Belief Mental Model
Sometimes a whole class is spent on the oral assessment.
It is okay to spend a whole class on fill-in-the blanks grammar exercises, but it’s not okay to spend a whole class practicing speaking and then being assessed on it.
Belief Mental Model
Grammar is not being assessed.
Grammar should be assessed by way of discrete grammar items.
Belief Mental Model
Example of assessment: Speaking
Students will be able to talk about past experiences related to music and expand the conversation by providing details on the experience.
Ask your partner some questions about musical experience. Begin with a “have you ever” question and then ask some follow-up questions. Below are some ideas, but you can create your own questions about experiences related to music.
Have you ever been to a rock/jazz/pop concert?
Have you ever played in a band?
Have you ever traveled just to go to a concert?
_____________________________________________________________?
Instructions
Students practice as much as they need until they feel ready;Teacher can:- Walk around the classroom
and assess pairs- Ask pairs to present to the
class when they are ready- Pairs record dialogues.- Teacher assesses students by
way of “real-time, almost surreptitious recording of student verbal and nonverbal behavior”(Brown, 2004. p. 267).
Assessment Course for Teachers
Assessment Consultants
Develop a growth mindset in teachers and students
Challenge mental models
Build a shared vision
Create a learning community
Managing change
CHANGE ONE TEACHER AND ONE STUDENT AT A TIME