managing a paradigm shift in assessment: a matter or mindset

Managing a paradigm shift in assessment – a matter of

mindset

Isabela Villas Boas, Casa Thomas Jefferson, Brasilia, Brazil

Grade oriented

Private school ranking

Public school annual evaluation

Culture of teaching to the test

Fierce competition to get into a good university – national examination

Casa Thomas Jefferson at a glance

Founded in 196316,000 + studentsSix branches + school contracts + in-

companyAround 200 teachersChildren - adultsBasic - post-advancedCourse Supervision – standardizationBalanced assessment system, but

with a greater emphasis on summative tests

The “Flex” courses

Around 3,000 Ss

Only adults

Four ten-week modules a year

Typical adult students

Traditional assessment system

After ten weeks of instruction:

One day for written test: reading comprehension, grammar, vocabulary; about 60% of the test focused on grammar structures and functions

One day for oral test: done in pairs; students stand outside the classroom and wait for their turn (following the proficiency test model)

How it all started - E-teacher course

Assessment

Summative and Formative Practices in Language Learning and Teaching

Traditional assessment system

Written test + oral test at the end of ten-week module

Ten weeks of instruction: accumulation of content

Focus on grammar and written exercises

Student anxiety

More selected-response items on the test

Lack of alignment

Negative washback effect

Summative Assessment

Formative Assessment

Traditional Assessment

Alternative Assessment

Aims of the new assessment system

Very little feedback Actionable feedback

Poor alignment Better alignment

New assessment system

1 2 3 4 5 6

Short assessments - 10 to 20 minutes2 oral assessments, necessarilyAt least 1 reading, 1 listening, and 1 writingMostly performance assessment of grammar

100 points

Last day of class: Final assessment daySs take the assessments they missed during the module or that they would like to retake (to improve outcomes)

PILOT

Traditional tests administered at the end to compare scores; comparison of final grades with typical student performance

Constant student feedback

Constant teacher feedback

Formal survey with students at the end

Ö Students had similar scores on their assessments and on the summative test

Ö Students had similar or higher final grades than the final grades in previous modules

Ö Students felt more at ease with the pilot system; less pressure

Building a shared vision - E-teacher course

Assessment

Summative and Formative Practices in Language Learning and Teaching

Phase-in stage

ImplementationWorkshops on

assessment

Ongoing feedback on

the outcomes of the new

system

Class observations

Revision of assessment

tools

Feedback from teachers and students at

the end of the module

New systemOld system

Students

More communication in classContent is not accumulatedNo need to memorize rulesNeed to study more frequently = more effective learningLess stressfulMore accurate assessment of the student because it is ongoingThe oral assessments are less stressfulNo last-minute studying onlyLighter“Forces” students to come to classMakes me feel more comfortable in class

Students

Fewer assessments

A final test is more effective in measuring knowledge

Have the six assessments + final test

Too easy

Fully Satisfactorily Partiallly No

Understand reasons for change

89% 6% 0% 0%

Support change 61% 17% 6% 0%

Teachers

No accumulation of content to assess.

Allows for remedial work and teacher self-assessment.

Student progress is more visible.

Immediate and personalized feedback to students.

Encourages more teacher reflection.

Students are assessed at their best.

Less stressful for students.

Possibility to re-teach and re-assess.

Teachers

Too many assessments.

Students keep asking if there will be an assessment the next class.

A little more time-consuming for teachers.

Difficulty with the oral assessments.

Two pilot groups

Two modules of each course

Four modules of each course

All levels: all teachers

The assessment system is appropriate for the students’ profile.

Totally Most of the time

Partially No No response

Change in staff

Few but very loud complaintsToo many assessments; too much time spent on assessments. Why not have four or three at the most?

Assessments are too easy. Students get higher grades than they used to; they won’t be ready for the next level.

Sometimes a whole class is spent on the oral assessment.

Grammar is not being assessed.

Our mental models – theories about the way the world works – influence our actions.

Examine mental models: Unexamined mental models limit people’s ability to change.

Schools that Learn, Peter Senge

Expecting an assessment will capture excellence, mediocrity, and failure as an educational measure reinforces a mindset that some students will always fail. That mindset has no place in our schools.

How Can Schools Develop Assessments That Matter?By Jill Berkowicz and Ann Myers, March 22, 2016Education Week

Too many assessments; too much time spent on assessments. Why not have three or four at the most?

It is okay to spend two full class days on oral and written tests, but it doesn’t seem okay to spend this same amount of time on smaller doses of assessment.

Belief Mental Model

It’s too easy. Students get higher grades than they used to; they won’t be ready for the next level.

Assessment as a student ranking tool;

Testing drives learning;

Quality = difficulty.

Belief Mental Model

Sometimes a whole class is spent on the oral assessment.

It is okay to spend a whole class on fill-in-the blanks grammar exercises, but it’s not okay to spend a whole class practicing speaking and then being assessed on it.

Belief Mental Model

Grammar is not being assessed.

Grammar should be assessed by way of discrete grammar items.

Belief Mental Model

Example of assessment: Speaking

Students will be able to talk about past experiences related to music and expand the conversation by providing details on the experience.

Ask your partner some questions about musical experience. Begin with a “have you ever” question and then ask some follow-up questions. Below are some ideas, but you can create your own questions about experiences related to music.

Have you ever been to a rock/jazz/pop concert?

Have you ever played in a band?

Have you ever traveled just to go to a concert?

_____________________________________________________________?

Instructions

Students practice as much as they need until they feel ready;Teacher can:- Walk around the classroom

and assess pairs- Ask pairs to present to the

class when they are ready- Pairs record dialogues.- Teacher assesses students by

way of “real-time, almost surreptitious recording of student verbal and nonverbal behavior”(Brown, 2004. p. 267).

Assessment Course for Teachers

Assessment Consultants

Develop a growth mindset in teachers and students

Challenge mental models

Build a shared vision

Create a learning community

Managing change

CHANGE ONE TEACHER AND ONE STUDENT AT A TIME