newsletter december 2017 - universidad veracruzana · 40 teasig member spotlight gladys quevedo...

TEASIG—EĞǁ ƐůĞƩ Ğƌ

Issue 63

TEASIG NEWSLETTERI s s u e 6 3D E C E M B E R 2 0 1 7

3 Editorial

4 >ĞƩ ĞƌĨƌŽŵƚŚĞŽŽƌĚŝŶĂƚŽƌƐ

5 dŚĞd^/' ĐŽŵŵŝƩ ĞĞ

6 t ĂƐŚďĂĐŬŽĨWƌĂĐƟĐĂůŶŐůŝƐŚĞǆĂŵŝŶĂƟŽŶƐŽŶƚŚĞƉƌŽĐĞƐƐŽĨƚĞĂĐŚŝŶŐĂŶĚůĞĂƌŶŝŶŐ Elżbieta Zawadowska-<ŝƩ Ğů

11 ĚƵĐĂƟŽŶŽƌĞǀ ĂůƵĂƟŽŶ Ilya Denisenko

15 D ĂǆŝŵŝƐŝŶŐĐŽůůĂďŽƌĂƟŽŶŝŶŽƌĂůŶŐůŝƐŚƚĞƐƟŶŐĂŶĞǆĂŵƉůĞĨƌŽŵŚŝŶĂ Susanna Wickes

19 ZĞĐĞƉƟǀ ŝƚǇƚŽůĞĂƌŶĞƌ-driven feedback Clare Maas

24 ' ŝǀ ŝŶŐƚĞƐƟŶŐƐŽŵĞŐŽŽĚƉƌĞƐƐĨŽƌĂĐŚĂŶŐĞ Assessment as learning check - ĂŵĞĂŶŝŶŐĨƵůƉĂƌĂĚŝŐŵƐŚŝŌĨŽƌƚƌĂŝŶĞĞƚĞĂĐŚĞƌƐ

Lynn Williams

28 ŶŵŝŚƵŵŝůĚĞŽƉŝŶŝſ Ŷ ϭ ůŝƐƚĞŶŝŶŐƚŽD ĞǆŝĐĂŶƐƚƵĚĞŶƚƐ ƉĞƌĐĞƉƟŽŶƐŽĨĂŶŶŐůŝƐŚlanguage proficiency test

David Ewing Ryan

35 IATEFL TEASIG Webinar(24 October 2017): Developing a test: Where do you start?When should you stop?

Jo Tomlinson

38 TEASIG Member Spotlight ŚƌŝƐƟŶĞŽŽŵďĞ

40 TEASIG Member Spotlight Gladys Quevedo Camargo

Price £ 4.50 Free for TEASIG members ISSN 2414-6307 www.iatefl.org

T h e N e w s l e t t e r o f t h e T e s t i n g , E v a l u a t i o n a n d A s s e s s m e n t S p e c i a l I n t e r e s t G r o u p

2


Issue 63 2

Editorial

t ĞůĐŽŵĞƚŽ/ƐƐƵĞϲϯŽĨƚŚĞd^/' EĞǁ ƐůĞƩ Ğƌ dŝŵĞŚĂƐƐŝŵƉůǇƐƉĞĚďǇƐŝŶĐĞŽƵƌůĂƐƚƉƵďůŝĐĂƟŽŶ ƚŚĞƵƚƵŵŶEĞǁ ƐůĞƩĞƌĂŶĚŝƚŝƐŝŶĚĞĞĚƚŚĂŶŬƐƚŽƚŚĞĐŽŶƚƌŝďƵƟŽŶƐŽĨŽƵƌŝŶĚƵƐƚƌŝŽƵƐd^/' ŵĞŵͲbers that we are able to bring out another News-ůĞƩĞƌƐŽƐŽŽŶ /ǁ ŽƵůĚůŝŬĞƚŽƚŚĂŶŬƚŚĞŵ ďŽƚŚTESIG members and non-members, for helping tomake this possible, and hope that they will inspireŵŽƌĞŽĨǇŽƵƚŽƐĞŶĚŝŶĂƌƟĐůĞƐ;ǁ ŚŝĐŚĚŽŶŽƚŶĞĞĚto be long) or take part in a TEASIG Member Spot-light interview. Further details can be found atŚƩ ƉƐ ƚĞĂ ŝĂƚĞŇŽƌŐ ŶĞǁ ƐůĞƩĞƌ

dŚŝƐŝƐƐƵĞƐƚĂƌƚƐŽī ǁ ŝƚŚƚŚƌĞĞĐŽŶƚƌŝďƵƟŽŶƐƐƵďͲŵŝƩ ĞĚďǇƉƌĞƐĞŶƚĞƌƐĂƚƚŚĞ/d&>ŽŶĨĞƌĞŶĐĞŝŶGlasgow last April. The first is based on a talk given at the joint ESPSIG/TEASIG Pre-Conference Event;WͿ /ŶƚŚŝƐĂƌƟĐůĞůǏ ďŝĞƚĂĂǁ ĂĚŽǁ ƐŬĂ-<ŝƩ ĞůĚĞͲscribes a new exam for BA students together with aƐƚƵĚǇǁ ŚŝĐŚǁ ĂƐƵƐĞĚƚŽŝŶǀ ĞƐƟŐĂƚĞŝƚƐǁ ĂƐŚďĂĐŬrecalling the power of tests to influence the teach-ing and learning process that precedes them. TheƐĞĐŽŶĚƚǁ ŽĂƌƟĐůĞƐĂƌĞďĂƐĞĚŽŶƉƌĞƐĞŶƚĂƟŽŶƐĂůƐŽgiven in Glasgow, both at the TEASIG day. Ilya Den-isenko’s thought-ƉƌŽǀ ŽŬŝŶŐĐŽŶƚƌŝďƵƟŽŶĞŶĚŽƌƐĞƐ ĂƵƚŚĞŶƟĐĂƐƐĞƐƐŵĞŶƚ ĂŶĚĞŶĐŽƵƌĂŐĞƐƚĞĂĐŚĞƌƐƚŽƐĞĞŽƉƉŽƌƚƵŶŝƟĞƐĂŶĚďĞŶĞĮ ƚƐŝƚƉƌĞƐĞŶƚƐ ǁ ŚŝůĞ^ƵƐĂŶŶĂt ŝĐŬĞƐ ĂƌƟĐůĞƉƌŽǀ ŝĚĞƐĂŶŝŶƚĞƌĞƐƟŶŐĂĐͲĐŽƵŶƚŽĨƚŚĞŝŶƚƌŽĚƵĐƟŽŶŽĨĂŐƌŽƵƉŽƌĂůƚĞƐƚŝŶŚŝͲŶĂ ĂůƐŽĐŽŶĐĞƌŶĞĚǁ ŝƚŚĂƵƚŚĞŶƟĐŵĞĂŶŝŶŐĨƵůĂƐͲsessment.

/ŶƚŚĞƌĞŵĂŝŶŝŶŐĐŽŶƚƌŝďƵƟŽŶƐ ůĂƌĞD ĂĂƐƐŚĂƌĞƐher experience of the LDF (Learner-Driven Feed-ďĂĐŬͿĂƉƉƌŽĂĐŚĂŶĚƐƚƵĚĞŶƚƐ Ăƫ ƚƵĚĞƐƚŽǁ ĂƌĚƐŝƚ The findings are promising, and Clare kindly pro-ǀ ŝĚĞƐƟƉƐĨŽƌƚƌǇŝŶŐŽƵƚƚŚŝƐƵƐĞĨƵůƚĞĐŚŶŝƋƵĞ>ǇŶŶt ŝůůŝĂŵƐƚĂŬĞƐĂƉŽƐŝƟǀ Ğǀ ŝĞǁ ŽĨĂƐƐĞƐƐŵĞŶƚŝŶŚĞƌwork as a teacher trainer, reminding us that as as-sessment is not necessarily a test and sharing someof the strategies she uses with her trainee teachers.dŽĐŽŶĐůƵĚĞƚŚŝƐƐĞĐƟŽŶ Ăǀ ŝĚǁ ŝŶŐZǇĂŶ ƐĂƌƟĐůĞmakes intriguing reading as he describes a study inD ĞǆŝĐŽƚŚĂƚŝŶǀ ĞƐƟŐĂƚĞĚƚŚĞĂƫ ƚƵĚĞƐĂŶĚŽƉŝŶŝŽŶƐŽĨƚŚĞŵŽƐƚŝŵƉŽƌƚĂŶƚ ďƵƚŽŌĞŶŶĞŐůĞĐƚĞĚ ƐƚĂŬĞͲ

holders in a language test, that is, the test-takersthemselves.

/ŶƚŚĞĮ ŶĂůƐĞĐƟŽŶŽĨƚŚĞEĞǁ ƐůĞƩĞƌ:ŽdŽŵůŝŶƐŽŶĂŶƐǁ ĞƌƐŽƵƚƐƚĂŶĚŝŶŐƋƵĞƐƟŽŶƐŝŶĂĨŽůůŽǁ -up to theŝŶĨŽƌŵĂƟǀ Ğǁ ĞďŝŶĂƌƐŚĞŐĂǀ ĞŽŶƚĞƐƚĚĞǀ ĞůŽƉŵĞŶƚon 24 October 2017. (Check out the next webinaron the TEASIG website!). We round off this issue by turning the TEASIG spotlight on two of our mem-ďĞƌƐĨƌŽŵĚŝī ĞƌĞŶƚĐŽƌŶĞƌƐŽĨƚŚĞŐůŽďĞŚƌŝƐƟŶĞCoombe, who is based in Dubai, and Gladys Queve-do Camargo from Brasilia.

Thanks again to everyone who contributed to thisEĞǁ ƐůĞƩĞƌĂŶĚĨŽƌƐŚĂƌŝŶŐƚŚĞŝƌŝĚĞĂƐǁ ŝƚŚƚŚĞTEASIG community. Very best wishes to you all forthe New Year – hoping to hear from you or see youat a TEASIG event in 2018!

Maggi Lussi Bell

TEASIG Joint Editor

3


Issue 63 3

Letter from the Coordinators

Dear TEASIG members

ƐƚŚŝƐŵĂǇǁ ĞůůďĞŵǇůĂƐƚůĞƩ ĞƌƚŽǇŽƵŝŶƚŚĞ

d^/' EĞǁ ƐůĞƩ ĞƌĂƐd^/' ŽŽƌĚŝŶĂƚŽƌEĞŝůŚĂƐ

asked me to write and say something about myself.

Some of you may have read in the latest issue of

ƚŚĞ/d&>ĞƵůůĞƟŶƚŚĂƚ/ŚĂǀ ĞďĞĞŶĞůĞĐƚĞĚƚŽ

represent all the 16 IATEFL SIGs on the Board of

Trustees. I feel very honoured to have been en-

ƚƌƵƐƚĞĚǁ ŝƚŚƚŚŝƐƉŽƐŝƟŽŶ ĂŶĚŚŽƉĞƚŚĂƚ/ĐĂŶůŝǀ Ğ

ƵƉƚŽƚŚĞĞǆƉĞĐƚĂƟŽŶƐƚŚĂƚŵǇĨĞůůŽǁ /' ŽŽƌĚŝͲ

nators have of me.

/ǁ ŝůůďĞĐŽŵĞ /' ZĞƉƌĞƐĞŶƚĂƟǀ ĞĨƌŽŵƉƌŝůϮϬϭϴĂƚ

the Annual Conference, which means that I will be

stepping down as TEASIG Coordinator. I will, how-

Ğǀ ĞƌƌĞŵĂŝŶŽŶƚŚĞd^/' ŽŵŵŝƩ ĞĞĂŶĚĐŽŶƟŶͲ

ƵĞƚŽǁ ŽƌŬǁ ŝƚŚD ĂŐŐŝŽŶƚŚĞEĞǁ ƐůĞƩ Ğƌ/ŚĂǀ Ğ

ƚǁ ŽƌĞĂƐŽŶƐĨŽƌǁ ĂŶƟŶŐǀ ĞƌǇŵƵĐŚƚŽĐŽŶƟŶƵĞ

with this work, connected with TEASIG itself and

ǁ ŝƚŚƉƵďůŝĐĂƟŽŶƐ

I became a member of IATEFL in 1980 because I

wanted to become part of a community of EFL

teachers. Almost the only contact with the associa-

ƟŽŶĂŶĚǁ ŝƚŚŽƚŚĞƌ/d&>ŵĞŵďĞƌƐǁ ĂƐƚŚƌŽƵŐŚ

ƉƌŝŶƚĞĚƉƵďůŝĐĂƟŽŶƐƐƵĐŚĂƐs ŽŝĐĞƐ ǁ ŚŝĐŚǁ ĂƐƐĞŶƚ

by post. I lurked happily in the background of

/d&>ĨŽƌĂǀ ĞƌǇůŽŶŐƟŵĞĨŽůůŽǁ ŝŶŐŝƚƐĂĐƟǀ ŝƟĞƐ

with great interest. However, for a number of per-

sonal and professional reasons, such as small chil-

ĚƌĞŶĂŶĚĨƌĞĞůĂŶĐĞǁ ŽƌŬ/ǁ ĂƐŶ ƚĂďůĞƚŽĂƩ ĞŶĚĂ

ĐŽŶĨĞƌĞŶĐĞƵŶƟůϮϬϬϱ ĂůƚŚŽƵŐŚŝƚŚĂĚďĞĞŶŽŶĞŽĨ

ŵǇĂŵďŝƟŽŶƐĨŽƌƐŽŵĞƟŵĞ/ǁ ĂƐǁ ŽƌŬŝŶŐĂƐĂ

ƚĞƐƚĚĞǀ ĞůŽƉĞƌĂŶĚŵǇƉƌĞƐĞŶƚĂƟŽŶǁ ĂƐƐĞůĞĐƚĞĚ

as part of the TEASIG Day, which turned out to be

ƚŚĞƐƚĂƌƚŽĨĂŚĂƉƉǇĂŶĚĨƌƵŝƞƵůƌĞůĂƟŽŶƐŚŝƉ ĂŶĚ

one which I now no longer want to forego.

Not only do I belong to the strange breed of test-

lovers, but I am a self-designated manic proof-

ƌĞĂĚĞƌ/ĚŝƐĐŽǀ ĞƌĞĚƚŚĞũŽǇƐŽĨĞĚŝƟŶŐĂŶĚƉƌŽŽĨ-

reading through my work as a test developer, and

my main professional responsibility is now quality

control of assessment procedures, which also in-

ǀ Žůǀ ĞƐĂŐƌĞĂƚĚĞĂůŽĨĞĚŝƟŶŐ/ĂĐƚƵĂůůǇĮ ŶĚŝƚŚĂƌĚ

ƚŽƌĞĂĚĂŶǇƚŚŝŶŐǁ ŝƚŚŽƵƚŵĞŶƚĂůůǇĞĚŝƟŶŐŝƚŽƌĮ ŶĚͲ

ŝŶŐƉƌŝŶƟŶŐŵŝƐƚĂŬĞƐ ďĞŚĂǀ ŝŽƵƌƐŽŵĞƟŵĞƐĚĞͲ

scribed as obsessive!

D ǇŝŶƚĞƌĞƐƚŝŶƉƵďůŝĐĂƟŽŶƐŝƐǁ ŚĂƚ/ĨĞĞů/ŚĂǀ Ğ

brought to TEASIG. TEASIG produces a regular

EĞǁ ƐůĞƩ ĞƌĂŶĚŚĂƐƉƌŽĚƵĐĞĚĂŶƵŵďĞƌŽĨŽŶĨĞƌͲ

ĞŶĐĞWƌŽĐĞĞĚŝŶŐƐĂƐǁ ĞůůĂƐĐŽŶƚƌŝďƵƟŶŐƚŽƚŚĞ

/d&>ĞƵůůĞƟŶĂŶĚs ŽŝĐĞƐ dŚĞƌĞĂƌĞŵĂŶǇŽƚŚĞƌ

ŝĚĞĂƐĨŽƌƉƵďůŝĐĂƟŽŶƐŝŶƚŚĞĂŝƌƐƵĐŚĂƐĂd^/'

peer-reviewed journal, a “Best of TEASIG” publica-

ƟŽŶ Ăd^/' ďůŽŐĂŶĚŵŽƌĞ, Žǁ Ğǀ ĞƌĂůůŽĨ

ƚŚĞƐĞĞǆĐĞůůĞŶƚŝĚĞĂƐƌĞƋƵŝƌĞŵŽƌĞƟŵĞĂŶĚĞŶĞƌŐǇ

ƚŚĂŶŽŶĞŽƌƚǁ ŽĐŽŵŵŝƩĞĞŵĞŵďĞƌƐĐĂŶƉƌŽǀ ŝĚĞ

So I am going to end this message with a call for

volunteers to work in the area of TEASIG publica-

ƟŽŶƐ ŶŽƚŽŶůǇǁ ŝƚŚŝĚĞĂƐĨŽƌĚĞǀ ĞůŽƉŵĞŶƚďƵƚĂůƐŽ

with a commitment to do something to make

d^/' ƐƐĞƌǀ ŝĐĞƐĞǀ ĞŶďĞƩĞƌĨŽƌĂůůŝƚƐŵĞŵďĞƌƐ

Please contact me if you wish to be a part of our

dynamic and progressive team.

ZĞŵĞŵďĞƌĞǀ ĞƌǇůŝƩ ůĞŚĞůƉƐ

Wishing you all the very best for

2018,

Judith

4


Issue 63 4

Mehtap InceEvents Coordinator

Neil BullockJoint Coordinator

Webinar Coordinator

Sharon HartleWebinar Coordinator

Thom KiddleWebmaster

Mehvar Ergun TurkkanMembership Officer

Maggi Lussi Bell:ŽŝŶƚEĞǁ ƐůĞƩ ĞƌĚŝƚŽƌ

Judith MaderJoint Coordinator

:ŽŝŶƚEĞǁ ƐůĞƩ ĞƌĚŝƚŽƌ

Ceyda MutluSocial Media Manager

Dave AllanD ĞŵďĞƌǁ ŝƚŚŽƵƚWŽƌƞŽůŝŽ

The TEASIG committee

dŽĮ ŶĚŽƵƚŵŽƌĞĂďŽƵƚd^/' ĐŽŵŵŝƩĞĞŵĞŵďĞƌƐŐŽƚŽŚƩ Ɖ ƚĞĂ ŝĂƚĞŇŽƌŐ ŝŶĚĞǆƉŚƉ ĂďŽƵƚ-teasig/teasig-ĐŽŵŵŝƩ ĞĞ

People involved in this issue:Editors: Maggi Lussi Bell, Judith MaderPhotos: TEASIG-CRELLA Conference, Luton 2017Design and Layout: elc – European Language Competence

ŚƩ Ɖ ƚĞĂ ŝĂƚĞŇ ŽƌŐ ŝŶĚĞǆ ƉŚƉ ĂďŽƵƚ-teasig/teasig-ĐŽŵ ŵŝƩ ĞĞ

http://tea.iatefl.org/index.php/about-teasig/teasig-committee/

5


Issue 63 5

ŽŶƚƌŝďƵƟŽŶƐƚŽƚŚĞd^/' E Ğǁ ƐůĞƩ Ğƌ

dŚĞ/d&>d^/' ;dĞƐƟŶŐǀ ĂůƵĂƟŽŶĂŶĚƐƐĞƐƐŵĞŶƚ ƉĞĐŝĂů/ŶƚĞƌĞƐƚ' ƌŽƵƉͿEĞǁ ƐͲůĞƩ ĞƌŝƐƉƵďůŝƐŚĞĚƚŚƌĞĞƟŵĞƐĂǇĞĂƌ– firstly, a post-IATEFL Annual Conference issue, between July and September, secondly, at the end of the calendar year, and finally, at the beginning of the calendar year, before the IATEFL Annual Conference. All issues are indigital form.

dŚĞŶĞǆƚƐƵďŵŝƐƐŝŽŶĚĂƚĞƐĨŽƌĂƌƟĐůĞƐĂŶĚĂĚǀ ĞƌƟƐĞŵĞŶƚƐĂƌĞϯϭ:ĂŶƵĂƌǇϮϬϭϴĂŶĚ31 May 2018..

ŽŶƚƌŝďƵƚŽƌƐĚŽŶŽƚŶĞĞĚƚŽďĞŵĞŵďĞƌƐŽĨ/d&>Žƌd^/' ŚŽǁ Ğǀ ĞƌĐŽŶƚƌŝďƵƟŽŶƐĨƌŽŵd^/' ĂŶĚ/d&>ŵĞŵďĞƌƐ ĂŶĚŽŶŵĂƩ ĞƌƐŽĨŝŶƚĞƌĞƐƚƚŽd^/' ŵĞŵďĞƌƐ ǁ ŝůůďĞgiven priority.

d^/' ƌĞƐĞƌǀ ĞƐƚŚĞƌŝŐŚƚƚŽĂĐĐĞƉƚŽƌƌĞũĞĐƚĂƌƟĐůĞƐĂƐŝƚĚĞĞŵƐŶĞĐĞƐƐĂƌǇ/ĨĂŶĂƌƟĐůĞŝƐĂĐĐĞƉƚĞĚĨŽƌƉƵďůŝĐĂƟŽŶ ŝƚŵĂǇďĞƐƵďũĞĐƚƚŽĞĚŝƟŶŐĂŶĚĂůƚĞƌĂƟŽŶƐŵĂǇďĞŵĂĚĞŝŶƟͲtles, headings, length and other aspects. Minor editorial changes in the text may be madefor reasons of space, style, clarity, acceptability and correctness of language. If more thanminor changes are considered necessary, the author(s) will be consulted by the Editors ofƚŚĞEĞǁ ƐůĞƩ ĞƌĨŽƌƚŚĞŝƌĂƉƉƌŽǀ Ăů

/ŶƐŽŵĞĐĂƐĞƐ ĂƌƟĐůĞƐŵĂǇŶŽƚďĞŝŶĐůƵĚĞĚŝŶƚŚĞŶĞǆƚƉƵďůŝƐŚĞĚŝƐƐƵĞŽĨƚŚĞd^/' EĞǁ ƐůĞƩ ĞƌďƵƚƌĞƐĞƌǀ ĞĚĨŽƌĂůĂƚĞƌŝƐƐƵĞdŚŝƐǁ ŝůůďĞĂƚƚŚĞƐŽůĞĚŝƐĐƌĞƟŽŶŽĨƚŚĞEĞǁ ƐͲůĞƩ ĞƌĚŝƚŽƌƐ

Guidelines for contributors

ƌƟĐůĞƐĚŽŶŽƚŶĞĞĚƚŽďĞůŽŶŐdŚĞǇŵĂǇďĞĂƐƐŚŽƌƚĂƐϲϬϬǁ ŽƌĚƐďƵƚƐŚŽƵůĚgenerally be no longer than 2000 words.

Photographs, graphics or diagrams should be sent as separate .jpg files. Please mark clearly in the text where they should be placed and ensure you have permission forƌĞƉƌŽĚƵĐƟŽŶŽĨĂŶǇƐƵĐŚŝƚĞŵƐ /ĨŝŶĚŽƵďƚ ƚŚĞEĞǁ ƐůĞƩ ĞƌĚŝƚŽƌƐŵĂǇĂƐŬĂĐŽŶƚƌŝďƵƚŽƌƚŽǀ ĞƌŝĨǇƚŚĞƐŽƵƌĐĞĂŶĚĂƵƚŚŽƌŝƐĂƟŽŶŽĨĂŶǇƐƵĐŚŵĂƚĞƌŝĂů

Only 5 key references should be given (where required). Other references should beavailable if requested.

WŝĞĐĞƐƐŚŽƵůĚŶŽƚŚĂǀ ĞďĞĞŶƉƵďůŝƐŚĞĚŽƌďĞƵŶĚĞƌŐŽŝŶŐĐŽŶƐŝĚĞƌĂƟŽŶĨŽƌƉƵďůŝĐĂͲƟŽŶĞůƐĞǁ ŚĞƌĞ/ĨƚŚŝƐŝƐƚŚĞĐĂƐĞƉůĞĂƐĞŝŶĐůƵĚĞŝŶĨŽƌŵĂƟŽŶŽŶǁ ŚĞƌĞĂŶĚǁ ŚĞŶƚŚĞĂƌƟĐůĞŚĂƐƉƌĞǀ ŝŽƵƐůǇďĞĞŶŽƌǁ ŝůůďĞƉƵďůŝƐŚĞĚ ĂŶĚĞŶƐƵƌĞǇŽƵŚĂǀ ĞƚŚĞƌŝŐŚƚƚŽƌĞƉƵďůŝƐŚĨƌŽŵƚŚĞĚŝƚŽƌŽĨƚŚĞŽƚŚĞƌƉƵďůŝĐĂƟŽŶ

D ŽƌĞŝŶĨŽƌŵĂƟŽŶŽŶŚŽǁ ǇŽƵĐĂŶĐŽŶƚƌŝďƵƚĞƚŽƚŚĞd^/' ŶĞǁ ƐůĞƩ ĞƌĐĂŶďĞĨŽƵŶĚŽŶŚƩ Ɖ ƚĞĂ ŝĂƚĞŇ ŽƌŐ ŝŶĚĞǆƉŚƉ ŶĞǁ ƐůĞƩ Ğƌ

http://tea.iatefl.org/index.php/newsletter/

6


Issue 63 6

Washback of Practical English examinationson the process of teaching and learning

Elżbieta Zawadowska-<ŝƩ Ğů

is currently working at the Uni-

versity of Social Sciences in

Warsaw. Her PhD thesis centred

on the washback of a school

leaving exam in English on the

teaching and learning process.

Her research interests include

ƚĞƐƟŶŐĂŶĚĂƐƐĞƐƐŵĞŶƚ ŝŶŶŽǀ ĂͲ

ƟŽŶƐŝŶǀ ĂƌŝŽƵƐĮ ĞůĚƐŽĨŵĞƚŚŽĚͲ

ology and the concept of

learning outcomes.

This paper is based on a presen-

ƚĂƟŽŶŐŝǀ ĞŶĂƚƚŚĞ^W/'

TEASIG Pre-Conference Event at

the IATEFL Conference in Glas-

gow in April 2017.

ϭ ǆĂŵŝŶWƌĂĐƟĐĂůŶŐůŝƐŚĂƐĂƌĞŇĞĐƟŽŶŽĨƚŚĞŝŵƉůĞŵĞŶƚĂƟŽŶŽĨƚŚĞE ĂƟŽŶĂůY ƵĂůŝĮ ĐĂƟŽŶƐFramework

dŚĞŝŶƚƌŽĚƵĐƟŽŶŽĨĂŶĞǁ ĞǆĂŵ

was connected with the implemen-

ƚĂƟŽŶŽĨƚŚĞEĂƟŽŶĂůY ƵĂůŝĮ ĐĂƟŽŶƐ

Framework (NQF) and new learning

outcomes for BA students at the

college where the research was

conducted. Before the change to

ƚŚĞŶĞǁ ĞǆĂŵ ĂƚƌĂĚŝƟŽŶĂůĞǆĂŵ

was used, composed of two parts:

ǁ ƌŝƩ ĞŶĂŶĚŽƌĂůdŚĞǁ ƌŝƩ ĞŶƉĂƌƚ

included reading comprehension,

ůŝƐƚĞŶŝŶŐĐŽŵƉƌĞŚĞŶƐŝŽŶ ǁ ƌŝƟŶŐ

and grammar competence. The oral

exam was taken in pairs and con-

sisted of a discussion on a given

subject. The format of the examina-

ƟŽŶŵĂƚĐŚĞĚƚŚĞǁ ĂǇƚŚĞĐůĂƐƐĞƐĂƚ

ƚŚĞĐŽůůĞŐĞǁ ĞƌĞĨŽƌŵĞĚ ĂƐWƌĂĐƟͲ

cal English classes were divided into

ĐŽŶǀ ĞƌƐĂƟŽŶĐŽŵďŝŶĞĚǁ ŝƚŚƌĞĂĚͲ

ing skills, audio-visual classes, gram-

ŵĂƌĂŶĚǁ ƌŝƟŶŐ

dŚĞŝŶƚƌŽĚƵĐƟŽŶŽĨƚŚĞE Y &ĐĂƚĂͲ

lysed the changes in the process of

teaching and learning. The classes

ĚĞǀ ŽƚĞĚƚŽƚĞĂĐŚŝŶŐWƌĂĐƟĐĂůŶŐͲ

lish (PE) were divided into blocks of

ŝŶƚĞŐƌĂƚĞĚƐŬŝůůƐĂŶĚ ĂĚĚŝƟŽŶĂůůǇ

ŝŶƚŽĐůĂƐƐĞƐĚĞǀ ŽƚĞĚƚŽƉƌŽĚƵĐƟǀ Ğ

skills. The new division is intended

to facilitate and accelerate the

achievement of learning outcomes,

especially as far as speaking and

ǁ ƌŝƟŶŐĂƌĞĐŽŶĐĞƌŶĞĚ

1.1. Washback of exams on the

process of teaching and learning

The research that has been con-

ducted so far (A. Hughes, J. Alder-

son, Li Cheng, E. Zawadowska-<ŝƩ ĞůͿ

shows that it is the manner of

ƚĞƐƟŶŐƚŚĂƚĐŚĂŶŐĞƐƚŚĞƉƌŽĐĞƐƐŽĨ

teaching and learning both as far as

ƉƌŝŽƌŝƟĞƐĂŶĚƚĞĐŚŶŝƋƵĞƐĂƌĞĐŽŶͲ

cerned.

This concept is also reflected in the

ƉƵďůŝĐĂƟŽŶ , Žǁ ƚŽƉƌĞƉĂƌĞĞĚƵĐĂͲ

ƟŽŶĂůƉƌŽŐƌĂŵŵĞƐŝŶĂĐĐŽƌĚĂŶĐĞ

with the NQF” by Andrzej Kraśniew-

ski (2011) who writes: “Proper plan-

ŶŝŶŐŽĨƚŚĞŵĞƚŚŽĚƐŽĨǀ ĞƌŝĮ ĐĂƟŽŶ

of learning outcomes may to a large

extent decide what outcomes have

been, in fact, achieved by the stu-

dent. This results from the fact that

ĂůĂƌŐĞƉĂƌƚĂŶĚƐŽŵĞƟŵĞƐƚŚĞŵĂͲ

jority of students adapts their way

of learning to the content and form

of tests. Thus, these are the meth-

ŽĚƐŽĨǀ ĞƌŝĮ ĐĂƟŽŶƐŽĨůĞĂƌŶŝŶŐŽƵƚͲ

comes which determine what stu-

dents know to a larger extent than

the teaching methods.”

The new exam was designed based

on this view. Summing up the differ-

ĞŶƚŽƉŝŶŝŽŶƐŽŶĂŶĚĚĞĮ ŶŝƟŽŶƐŽĨ

washback, washback may be de-

fined as the influence of exams, in-

ƚĞŶƟŽŶĂůŽƌƵŶŝŶƚĞŶƟŽŶĂůďŽƚŚ

ƉŽƐŝƟǀ ĞĂŶĚŶĞŐĂƟǀ ĞŽŶƚŚĞůĞĂƌŶͲ

ing and teaching process, as this is

widely understood, and on all its

ƉĂƌƟĐŝƉĂŶƚƐĂƐǁ ĞůůĂƐŽŶƚŚĞĞŶƟƌĞ

ĞĚƵĐĂƟŽŶĂůƐǇƐƚĞŵ; Ăǁ ĂĚŽǁ ƐŬĂ

7


Issue 63 7

-<ŝƩ ĞůϮϬϭϯ Ϳ /ŶƚŚĞƉƌĞƐĞŶƚƉĂƉĞƌ

two major aspects of the washback

of the PE exam will be presented. It

is worth adding that in order to

ĂĐŚŝĞǀ ĞƉŽƐŝƟǀ Ğǁ ĂƐŚďĂĐŬƐƚƵĚĞŶƚƐ

need to be acquainted with the pur-

pose of the exam and regard its re-

sults as credible and fair, and they

also need to receive a detailed re-

port on their exam results, rather

than only a mark or a percentage

ƐĐŽƌĞĚĚŝƟŽŶĂůůǇƚŚĞƚĂƐŬƐƐŚŽƵůĚ

ďĞĂƵƚŚĞŶƟĐ

2. The concept of the PE exam

The new concept of the exam

ƐƚƌĞƐƐĞƐƚŚĞƌŽůĞŽĨƉƌŽĚƵĐƟǀ ĞƐŬŝůůƐ

in the teaching and learning process

and for this reason both speaking

ƐŬŝůůƐĂŶĚǁ ƌŝƟŶŐƐŬŝůůƐĂƌĞƚĞƐƚĞĚ

ƚǁ ŝĐĞ/ŶĐĂƐĞŽĨǁ ƌŝƟŶŐƐŬŝůůƐ ƚŚĞ

progress students have made during

the whole semester is assessed (by

ŵĞĂŶƐŽĨĂƉŽƌƞŽůŝŽͿ ǁ ŚŝĐŚƐƵƉͲ

ports both autonomy and system-

ĂƟĐůĞĂƌŶŝŶŐĂŶĚĂůƐŽŚĞůƉƐƚŽŝŶͲ

ĐƌĞĂƐĞŵŽƟǀ ĂƟŽŶĨŽƌůĞĂƌŶŝŶŐĂƐ

frequent feedback is given to stu-

ĚĞŶƚƐ ůƐŽ ĐƌŝƟĐĂůƚŚŝŶŬŝŶŐĂďŝůŝƚǇ

and the ability to express opinions

ŝŶǁ ƌŝƟŶŐŝƐƚĞƐƚĞĚ;ǁ ŝƚŚƌĞĨĞƌĞŶĐĞ

to reading).

Speaking skills are also assessed

twice – Į ƌƐƚ ŝŶĐŽŶŶĞĐƟŽŶǁ ŝƚŚůŝƐͲ

tening skills to increase the authen-

ƟĐŝƚǇŽĨƚŚĞƚĞƐƚ ĂƐŝŶƌĞĂůůŝĨĞ

ƐƉĞĂŬŝŶŐŝƐŵŽƐƚŽŌĞŶĐŽŶŶĞĐƚĞĚ

with listening, and then for the sec-

ŽŶĚƟŵĞŝŶĐŽŶŶĞĐƟŽŶǁ ŝƚŚĂ

WŽǁ ĞƌWŽŝŶƚƉƌĞƐĞŶƚĂƟŽŶĂŶĚĚŝƐͲ

cussion of a selected subject (as the

ĂďŝůŝƚǇƚŽƐƉĞĂŬŝŶƉƵďůŝĐŝƐŽŌĞŶ

expected of the students by pro-

ƐƉĞĐƟǀ ĞĞŵƉůŽǇĞƌƐͿ

dŚĞŶĞǁ ĨŽƌŵĂƚŽĨƚŚĞĞǆĂŵŝŶĂƟŽŶ

makes use of modern teaching tech-

niques – film, IT and also (thanks to

ƚŚĞƉŽƌƞŽůŝŽĂŶĚƉƌĞƐĞŶƚĂƟŽŶƐͿƌĞͲ

duces to some extent the risk of

ƵŶƌĞůŝĂďůĞƚĞƐƟŶŐĂƐ ŝĨŽŶůǇŽŶĞƉĞƌͲ

formance is assessed, the quality of

this may be affected by, for in-

stance, stress. The new format of

the exam has been evaluated both

by students and academic teachers

ŝŶƐƉĞĐŝĂůůǇĐŽŶƐƚƌƵĐƚĞĚƋƵĞƐƟŽŶͲ

naires.

dŚĞƌĞƐƵůƚƐŽĨƚŚĞƐĞƋƵĞƐƟŽŶŶĂŝƌĞƐ

will be used to modify both the con-

cept of the exam and the way it is

graded. The conclusions may serve

to change the concept of teaching

ƚŚƌŽƵŐŚƚŚĞŵĞƚŚŽĚƐŽĨǀ ĞƌŝĮ ĐĂƟŽŶ

of teaching concepts.

3. The study

dŚĞĂŝŵŽĨƚŚĞƐƚƵĚǇŝƐƚŽŝŶǀ ĞƐƟͲ

gate the washback of a new exam

on students and teachers, and find

out their opinion of the test format.

ϯ ϭ dŚĞƋƵĞƐƟŽŶŶĂŝƌĞ

dŚĞƐƵƌǀ ĞǇƋƵĞƐƟŽŶŶĂŝƌĞǁ ĂƐĚĞǀ ĞůͲ

oped specifically for the study and

administered in Polish. Nineteen of

ƚŚĞƋƵĞƐƟŽŶƐǁ ĞƌĞĐůŽƐĞĚƋƵĞƐƟŽŶƐ

and one open-ended, the aim of

which was to allow the respondents

to voice their opinions of the new

exam in a more precise way. The

surveys of teachers and students

ǁ ĞƌĞĂƐŝĚĞŶƟĐĂůĂƐƉŽƐƐŝďůĞŝŶ

terms of their content.

3.2. Profile of the respondents

A total of 70 valid responses from

the students of philology at a pri-

vate university and 10 from their

teachers were received from those

respondents who volunteered to

take part in the study. All the stu-

dents are 1st year BA students; 70%

of them study English as their ma-

jor. More than half of the teachers

teach English and almost all are in-

ƚĞƌĞƐƚĞĚŝŶƚĞƐƟŶŐŝƐƐƵĞƐ dŚĞŝƌ

teaching experience ranges from 2

ƚŽϭϬǇĞĂƌƐ , Žǁ Ğǀ ĞƌŶŽĐŽƌƌĞůĂƟŽŶ

between their opinions and experi-

ence has been found.

4. Discussion of results

A comparison of teachers’ and stu-

dents’ most significant answers to

ƚŚĞƋƵĞƐƟŽŶƐŝŶƚŚĞƐƵƌǀ ĞǇŝƐƉƌĞͲ

sented in Table 1.

Table 1 Comparison of opinions

“The new concept ofthe exam stresses the

role of productive skills inthe teaching and

learning process and forthis reason both speak-ing skills and writing skills

are tested twice.”

8


Issue 63 8

The majority of both teachers and

students realize that the manner of

ƚĞƐƟŶŐŝŶŇƵĞŶĐĞƐƚŚĞƉƌŽĐĞƐƐŽĨ

teaching and learning, which is re-

ŇĞĐƚĞĚŝŶƚŚĞĂĐƟǀ ŝƟĞƐŵŽƐƚĨƌĞͲ

quently used during the classes.

dŚĞƚĂďůĞĚŽĞƐŶŽƚŝŶĐůƵĚĞĂĐƟǀ ŝƟĞƐ

perceived as most frequent by less

than 10% of the respondents

(listening to CDs, projects, transla-

ƟŽŶƐͿ dŚĞƚĂďůĞƐŚŽǁ ƐƚŚĂƚƚĞĂĐŚͲ

ĞƌƐĨŽĐƵƐŽŶĂĐƟǀ ŝƟĞƐƚŚĂƚĂƌĞƐŝŵŝͲ

ůĂƌŽƌŝĚĞŶƟĐĂůƚŽƚŚĞĞǆĂŵŝŶĂͲ

ƟŽŶƚĂƐŬƐ WŽǁ ĞƌWŽŝŶƚƉƌĞƐĞŶͲ

ƚĂƟŽŶƐƐĞĞŵƚŽďĞƚŚĞĂĐƟǀ ŝƚǇ

perceived as even more fre-

quently used by students than

by the teachers. It is noteworthy

ƚŚĂƚŶĞǁ ŵĞƚŚŽĚƐŽĨƚĞƐƟŶŐ

ƌĞĐĞŝǀ ĞĚŚŝŐŚƌĂƟŶŐƐĨƌŽŵďŽƚŚ

teachers and students.

ZĞƐƉŽŶƐĞƐƚŽƚŚĞƋƵĞƐƟŽŶĐŽŶͲ

cerning the use of L1 may sug-

gest that the format of the ex-

am is too complicated and leads

to overuse of L1 on the part of

the teachers, who explain the

format of the exam and its rules

in L1 and this, in turn, provokes

students to use L1 as well. It

must be noted, however, that

talking about the exam is not

ƚŚĞŽŶůǇĐůĂƐƐƌŽŽŵƐŝƚƵĂƟŽŶŝŶ

which L1 is used, so it is rather a

ŵĂƩĞƌŽĨƚĞĂĐŚĞƌƐ ŚĂďŝƚƐĂŶĚ

these may gradually change.

The teachers are of the opinion

that learners made the most

ƐŝŐŶŝĮ ĐĂŶƚƉƌŽŐƌĞƐƐŝŶƌĞĐĞƉƟǀ Ğ

skills: listening and reading, as

well as vocabulary, which is di-

ƌĞĐƚůǇĐŽŶŶĞĐƚĞĚǁ ŝƚŚƚŚĞŝƌĞƐƟͲ

mate of their achievements in

these skills. Progress in speaking

ĂŶĚǁ ƌŝƟŶŐŝƐůĞƐƐƐƉĞĐƚĂĐƵůĂƌ

even though the aim of the changes

ǁ ĂƐƚŚĞĚĞǀ ĞůŽƉŵĞŶƚŽĨƉƌŽĚƵĐƟǀ Ğ

skills. The least progress was

achieved in grammar competence,

which most probably results from

the fact that grammar is not sepa-

9


Issue 63 9

rately tested on the exam. The stu-

dents share the teachers’ opinion

that they made the most significant

ƉƌŽŐƌĞƐƐŝŶƌĞĐĞƉƟǀ ĞƐŬŝůůƐ ůŝƐƚĞŶŝŶŐ

and reading, as well as in vocabu-

ůĂƌǇĂĐƋƵŝƐŝƟŽŶ ŝŵŝůĂƌůǇƚŽƚŚĞ

teachers, they feel they made less

ƉƌŽŐƌĞƐƐŝŶƐƉĞĂŬŝŶŐǁ ƌŝƟŶŐĂŶĚŝŶ

grammar competence.

/ƚŝƐŝŶƚĞƌĞƐƟŶŐƚŚĂƚƚĞĂĐŚĞƌƐĐŽŶͲ

sider students’ progress to be

higher than the students them-

ƐĞůǀ ĞƐĚŽ ǁ ŚŽƐĞĞƐƟŵĂƚĞŝƐ

closer to the actual results of

the exam. It seems likely that

teachers observe students’

achievements from the per-

ƐƉĞĐƟǀ ĞŽĨƚŚĞǁ ŚŽůĞĂĐĂĚĞŵŝĐ

year and students see the quali-

ƚǇŽĨƉĂƌƟĐƵůĂƌƉĞƌĨŽƌŵĂŶĐĞƐŽƌƚŚĞ

results of individual tests. The actu-

ĂůŐƌĂĚĞƐĂƌĞůŽǁ ĞƌĞĚďǇƵŶƐĂƟƐĨĂĐͲ

tory marks.

Some discrepancies exist in the

teachers’ and students’ views on

the pros and cons of the exam.

dĞĂĐŚĞƌƐƉŽŝŶƚŽƵƚƚŚĂƚƉŽƌƞŽůŝŽƐ

ĂƌĞƟŵĞ-consuming and in the stu-

dents’ opinion, make the exam eas-

ier to pass. Teachers complain

about the fact that grammar is not

tested and that accuracy is not

sufficiently stressed in the criteria,

whereas students approve of the

fact that the exam focuses on com-

ŵƵŶŝĐĂƟŽŶ

Consequently, teachers and stu-

dents share the view that exam

ƚĂƐŬƐƌĞƐĞŵďůĞƐŝƚƵĂƟŽŶƐĞŶĐŽƵŶͲ

tered in real life and may be regard-

ĞĚĂƐĂƵƚŚĞŶƟĐdŚĞǇĂƌĞĂůƐŽŽĨ

the opinion that the exam is valid

and more just.

Conclusions

As it turns out, the concept of

washback of important exams on

the teaching and learning process is

no less true in case of the PE exam

that has been discussed here. The

ŵŽƐƚŝŵƉŽƌƚĂŶƚƉŽƐŝƟǀ Ğǁ ĂƐŚďĂĐŬ

is that teachers focus more on pro-

ĚƵĐƟǀ ĞƐŬŝůůƐ ĂƐŝŶĐůĂƐƐƚŚĞǇŵŽƐƚ

ĨƌĞƋƵĞŶƚůǇƵƐĞĂĐƟǀ ŝƟĞƐƌĞƐĞŵďůŝŶŐ

ƚŚĞĞǆĂŵŝŶĂƟŽŶƚĂƐŬƐ , Žǁ Ğǀ Ğƌ

progress in those skills is not auto-

ŵĂƟĐĂůůǇŚŝŐŚĞƌƚŚĂŶŝŶƌĞĐĞƉƟǀ Ğ

skills, an aspect which needs to be

improved. Insufficient progress may

result from the overuse of L1 by

teachers, something which needs

ƚŽďĞĐŚĂŶŐĞĚ EĞŐůĞĐƟŶŐĂĐĐƵƌĂĐǇ

seems to be another important

problem, which is why including

grammar tasks in a new exam is

postulated by the teachers. Most

probably, the students will not ap-

ƉƌŽǀ ĞŽĨƚŚĂƚƐŽůƵƟŽŶ ĂƐƚŚĞǇǀ ĂůƵĞ

ŵŽƐƚƚĂƐŬƐƐƵĐŚĂƐƚŚĞƉŽƌƞŽůŝŽĂŶĚ

ƉƌĞƐĞŶƚĂƟŽŶ ǁ ŚŝĐŚƐĞĞŵƚŽƚŚĞŵ

ĨĂƌƐĂĨĞƌƚŚĂŶƚƌĂĚŝƟŽŶĂůƚĞƐƚƐ K Ŷ

the whole, the new exam is ap-

proved of both by the students and

the teachers as more valid, reliable

and contemporary. The teachers

ĂĚĚŝƟŽŶĂůůǇǀ ĂůƵĞƚŚĞƉŽƐƐŝďŝůŝƚǇŽĨ

Ğǀ ĂůƵĂƟŽŶŽĨƚŚĞĞǆĂŵ &ŽƌƚŚŝƐ

ƌĞĂƐŽŶĂŶĞǁ ƐƚƵĚǇŝƐƉůĂŶŶĞĚĂŌĞƌ

the changes in the exam are intro-

duced.

References

Alderson, J.C./ Wall, D (1993), Does

washback exist? (w:) “Applied Linguis-

ƟĐƐ 14, 115-129.

ŚĞŶŐ> t ĂƚĂŶĂďĞz ƵƌƟƐ

(2004). t ĂƐŚďĂĐŬŝŶůĂŶŐƵĂŐĞƚĞƐƟŶŐ

Research context and methods.

Mahwa.

Hughes, A (1994). dĞƐƟŶŐĨŽƌůĂŶŐƵĂŐĞ

teachers. Cambridge.

Kraśniewski, A (2011). Jak przygotować

programy kształcenia zgodnie z

wymaganiami Krajowych Ram

Kwalifikacji dla Szkolnictwa Wyższego?

Warszawa.

Zawadowska-<ŝƩ Ğů ;ϮϬϭϯ Ϳ Nowa

matura z języków obcych: szanse i

zagrożenia. Efekt zwrotny egzaminu

z języka angielskiego. Piaseczno.

“It is interesting thatteachers consider students’progress to be higher than

the students themselves do,whose estimate is closer to

the actual results of theexam.”

10


Issue 63 10

IATEFL Testing, Evaluation and AssessmentSpecial Interest Group (TEASIG)

Pre-Conference Event9 April 2018

IATEFL Annual Conference, Brighton

Assessing Listening Why do we want to test listening? What listening skills do we want to test and how? , Žǁ ĐĂŶǁ ĞďĞƩĞƌŝĚĞŶƟĨǇĂŶĚĂƐƐĞƐƐƚŚĞƐĞƐŬŝůůƐ Where can we source appropriate material? Should it be scripted or taken from the real world? t ŚĂƚŝƐƚŚĞŝŵƉŽƌƚĂŶĐĞŽĨƐƚĂƟƐƟĐƐŝŶƚŚĞĂƐƐĞƐƐŵĞŶƚŽĨůŝƐƚĞŶŝŶŐƐŬŝůůƐ

In this 2-ƉĂƌƚWǁ Ğǁ ŝůůďĞĂƩ ĞŵƉƟŶŐƚŽƚĂŬĞƚŚĞůŝĚŽī ƚŚŝƐŵǇƐƚĞƌŝŽƵƐǁ ŽƌůĚǁ ŝƚŚƚŚĞŚĞůƉŽĨĞǆƉĞƌƚƐĂŶĚƉƌĂĐƟƟŽŶĞƌƐŝŶƚŚŝƐĚŽŵĂŝŶ

dŚĞWŝƐĂŝŵĞĚĂƚĂůůƚĞĂĐŚĞƌƐ ǁ ŚĞƚŚĞƌƚŚĞǇĂƌĞŶĞǁ ƚŽƚŚŝƐĂƐƉĞĐƚŽĨƚĞƐƟŶŐĞǀ ĂůƵĂƟŽŶĂŶĚĂƐƐĞƐƐŵĞŶƚŽƌǁ ŝƐŚƚŽƌĞĨƌĞƐŚƚŚĞŝƌŬŶŽǁ ůĞĚŐĞĂŶĚƉƵƚŝƚƚŽƉƌĂĐƟĐĂůƵƐĞdŚĞƌĞǁ ŝůůĂůƐŽďĞƌŽŽŵĨŽƌƋƵĞƐƟŽŶƐƚŽĞǆƉĞƌƚƐĂŶĚĚŝƐĐƵƐƐŝŽŶŽĨďĞƐƚƉƌĂĐƟĐĞ

Morning – Plenary Speakers

John FieldThe University of Bedfordshire, UK

ŽŐŶŝƟǀ ĞƉƌŽĐĞƐƐĞƐŝŶůŝƐƚĞŶŝŶŐ

Sheila ThornThe Listening Business

ŶĞǁ ŵĂƚƌŝǆĨŽƌƚŚĞƚĞƐƟŶŐŽĨůŝƐƚĞŶŝŶŐ

Rita GreenTest Development Training & Analysis Ltd ƚĂƟƐƟĐĂůĞǀ ŝĚĞŶĐĞƚŽƐƵƉƉŽƌƚĂƐƐĞƐƐŵĞŶƚŽĨůŝƐƚĞŶŝŶŐ

ŌĞƌŶŽŽŶ– Hands-on workshopsůĞĚďǇE />ĂŶĚ>dϭϮϯǁ ŝƚŚŽƉƉŽƌƚƵŶŝƟĞƐƚŽŵĂƚĐŚƚŚĞƚŚĞŽƌǇĂŶĚŝĚĞĂƐƚŽǇŽƵƌǁ ŽƌůĚ

For more information and registration

https://tea.iatefl.org/upcoming-teasig-events/

https://secure.iatefl.org/registration/conf_reg_login.php

This event is being generously sponsored by Cambridge English Language Assessment.

https://tea.iatefl.org/upcoming-teasig-events/

https://secure.iatefl.org/registration/conf_reg_login.php

11


Issue 63 11

Education or evaluation?

Imagine that a graduate English ex-

am is obligatory for all your stu-

dents and it is taking place tomor-

row. Will they pass? Imagine your

brightest students, will they have

any problems? Think of the student

that worries you most. Do you think

they will be lucky enough to get

through the test with mostly correct

answers? Is it the speaking and

ǁ ƌŝƟŶŐƉĂƌƚƐƚŚĂƚĐŽŶĐĞƌŶǇŽƵ Ž

you agree that what we teach, what

students need, and what is being

tested are the same thing? When a

student gets top marks in an exam,

does it mean they are well prepared

for life and their career?

t Ğůůǁ ĞƐƟůůŚĂǀ ĞŵĂŶǇƋƵĞƐƟŽŶƐ

ĂďŽƵƚĞǀ ĂůƵĂƟŽŶ D ŽƐƚƚĞĂĐŚĞƌƐ

usually agree that students should

be taught relevant skills but others

may claim that they themselves lack

the tools for developing such skills.

Normally, an EL teacher has to up-

grade students’ grammar and vo-

cabulary on different topics as well

as develop students’ skills. Further-

ŵŽƌĞƐƵďƐĞƋƵĞŶƚĞǆĂŵŝŶĂƟŽŶƐǁ ŝůů

add some more uncertainty, so the

task formats used in exams should

be explained and thoroughly in-

ƐƟůůĞĚ D ĂǇďĞĂůůƚŚŝƐĐƌŝƟĐĂůƚŚŝŶŬͲ

ŝŶŐ/dĂŶĚĐŽŵŵƵŶŝĐĂƟǀ ĞĐŽŵƉĞͲ

ƚĞŶĐĞƐŚŽƵůĚďĞůĞŌƚŽƐŽŵĞďŽĚǇ

ĞůƐĞďĞĐĂƵƐĞǁ ĞĚŽŶ ƚŚĂǀ ĞƟŵĞĨŽƌ

it?

Can you put an equals sign between

what you teach, what your students

need, and what the exam will test?

If you consider these competences

important, you no doubt spend con-

ƐŝĚĞƌĂďůĞƟŵĞĂŶĚĞī ŽƌƚŽƌŐĂŶŝƐŝŶŐ

ƉƌĂĐƟĐĞŽĨƚŚĞŵ zŽƵŵŝŐŚƚďĞ

struggling with the syllabus, trying

ƚŽĮ ŶĚĞŶŽƵŐŚƟŵĞĂŶĚƐĞĂƌĐŚŝŶŐ

for sufficient materials. You do it

simply because there is a real world

outside the classroom and correct

grammar alone won’t suffice. We,

as teachers, just need to suggest

something more applicable to office

or freelance work in business.

School is not just a place where

knowledge and skills are learned.

Every subject should contribute to

cross-subject competence building –

‘should’, but doesn’t ‘have to’. How-

Ğǀ ĞƌƚŚĞƌĞĂƌĞƐƟůůƚŚŝŶŐƐƚŚĂƚĂƌĞĂ

ŵƵƐƚ ƚŚĞƐǇůůĂďƵƐ ĞĚƵĐĂƟŽŶĂůƉƌŽͲ

gramme, year plan, and even the

teacher’s book guide.

So, will an EL teacher risk the lim-

ited hours available on building up

language skills to develop some-

thing that neither a test nor an ex-

am can really assess? What if they

ĚŽĮ ŶĚƚŚĞĞǆƚƌĂƟŵĞ

Ilya Denisenko

Ilya Denisenko is Head of theLanguages Department at theState Academic University for, ƵŵĂŶŝƟĞƐ;^h , D ŽƐĐŽǁ Russia) and a founder of Au-ƚŚĞŶƟĐ>dƐƐŽĐŝĂƟŽŶ dŚŝƐyear he has completed a studyon assessment and given atalk at the IATEFL Conference.

This paper is based on aƉƌĞƐĞŶƚĂƟŽŶŐŝǀ ĞŶĂƚƚŚĞTEASIG Day at the IATEFL Con-ference in Glasgow in April2017.

12


Issue 63 12

English as a subject makes it possi-

ble to speak about anything from

ancient history to space flight; it

ĚŽĞƐŶ ƚŚĂǀ ĞĐŽŶƚĞŶƚƌĞƐƚƌŝĐƟŽŶƐ

(that depends on the cultural tradi-

ƟŽŶƐŽĨǇŽƵƌĐŽƵŶƚƌǇŽĨĐŽƵƌƐĞͿ

This means that an EL lesson can be

built on various contexts, related to

interests, experience and the pro-

ĨĞƐƐŝŽŶĂůŽƌĨƵƚƵƌĞĞĚƵĐĂƟŽŶĂů

needs of the students. This objec-

Ɵǀ ĞŵĂŬĞƐĂŶŶŐůŝƐŚůĞƐƐŽŶ

unique in that it prepares stu-

dents for real-life problems

ŵƵĐŚďĞƩĞƌƚŚĂŶĂŶǇƚŚŝŶŐĞůƐĞ

in the school curriculum.

An English lesson, moreover, is

ĂŶŝĚĞĂůƟŵĞĂŶĚƉůĂĐĞƚŽďƵŝůĚ

up the crucial skills needed to-

day. A teacher doesn’t have to

ƌĞĚƵĐĞŽƌĐĂŶĐĞůƚƌĂĚŝƟŽŶĂů

ŐƌĂŵŵĂƌŽƌǀ ŽĐĂďƵůĂƌǇƉƌĂĐƟĐĞ

(if they can’t do without it) to

ƐƉĂƌĞƐŽŵĞƟŵĞĨŽƌĂŶŽƚŚĞƌƚĂƐŬ

dŚĞƌĞŝƐĂŶŽƚŚĞƌƐŽůƵƟŽŶ

There is no need to go too deeply

ŝŶƚŽƚŚĞĐĂůĐƵůĂƟŽŶƐ ƚŚĞůĂƌŐĞƐƚ

ĂŵŽƵŶƚŽĨƟŵĞƐƉĞŶƚŝŶĞǀ ĞƌǇĐůĂƐƐͲ

room around the world is used for

Ğǀ ĂůƵĂƟŽŶ dŚĞƟŵĞ>ƚĞĂĐŚĞƌƐ

ƐƉĞŶĚŽŶƚĞƐƟŶŐĐĂŶďĞƵƐĞĚŵŽƌĞ

efficiently if it’s used for teaching

purposes as well.

Assessment is a valuable part of the

ĞĚƵĐĂƟŽŶĂůƉƌŽĐĞƐƐƚŚĂƚŝƐŽŌĞŶŶŽƚ

used enough. Many scholars have

agreed that assessment has a dou-

ble-duty and works as a scaling and

a teaching tool. However, many

ƉƌĂĐƟĐŝŶŐƚĞĂĐŚĞƌƐĮ ŶĚŝƚƚĂŬĞƐƚŽŽ

ŵƵĐŚĞī ŽƌƚƚŽƐƉĞŶĚĂůŽƚŽĨƟŵĞ

Ğǀ ĂůƵĂƟŶŐƚŚĞŝƌƐƚƵĚĞŶƚƐ ĂƚůĞĂƐƚ

ĨŽƌŵĂƟǀ ĞůǇ/ŶĂĚĚŝƟŽŶ ŵŽƐƚƚĞĂĐŚͲ

ĞƌƐĚŽŶ ƚƚƌǇƚŽĚĞǀ ĞůŽƉƚĞƐƟŶŐŵĂͲ

terials themselves. A well-organised

assessment system could solve such

problems.

Imagine such a system exists. What

ĂƌĞŝƚƐĐŚĂƌĂĐƚĞƌŝƐƟĐƐ &ŝƌƐƚŽĨĂůůŝƚ

should be built on students’ perfor-

mance. This means that students

need to be evaluated on how they

ĚĞĂůǁ ŝƚŚĂƌĞĂůƉƌĂĐƟĐĂůƚĂƐŬ/ŶƚŚŝƐ

ĐĂƐĞŽďƐĞƌǀ ĂƟŽŶŽĨƚŚĞƐƚƵĚĞŶƚƐ

ǁ ŽƌŬƐŚŽǁ ƐƌĞĂůĂĐƋƵŝƐŝƟŽŶŽĨĐŽŵͲ

petences.

Another vital thing is to make per-

formance meaningful for students,

which can be done through the con-

text of a task. For this reason, tasks

should be correlated with the stu-

ĚĞŶƚƐ ƉŽƚĞŶƟĂůŬŶŽǁ ůĞĚŐĞĂŶĚƉƌŽͲ

vide them with relevant feedback.

D ĞĂŶŝŶŐĨƵůŵĞĂŶƐŝŶƚĞƌĞƐƟŶŐĐŽŶͲ

nected with their experience, as

ǁ ĞůůĂƐƚŚŽƵŐŚƚĂŶĚĐƌĞĂƟǀ ŝƚǇƉƌŽͲ

voking.

Such meaningful contexts bolster

ŵŽƟǀ ĂƟŽŶ t ŚĞŶĂƐƚƵĚĞŶƚĐĂŶ

apply their own personal experi-

ĞŶĐĞŝƚƉƌŽǀ ŝĚĞƐƐĐŽƉĞĨŽƌĐƌĞĂƟǀ ŝƚǇ

and readiness to work in the pro-

ƉŽƐĞĚƐĞƫ ŶŐ

I’d like to illustrate this with one

example. A school boy was consid-

ered to be a poorly performing pupil

by all his teachers. Even those who

didn’t actually teach him were

Ăǁ ĂƌĞŽĨŚŝƐůŽǁ ƉŽƚĞŶƟĂů, ĞŽďǀ ŝͲ

ously behaved in a way that was

consistent with how he was seen,

and his teachers usually didn’t ex-

pect much of him. Yet, on an English

ƉƌŽũĞĐƚĂďŽƵƚĐĞƌƚĂŝŶƚŽƵƌŝƐƚĂƩ ƌĂĐͲ

ƟŽŶƐŚĞǁ ĂŶƚĞĚƚŽƐĞĞŝŶŚŝƐĂƌĞĂ

he really did his best. It turned out

that he was into extreme sports, for

which there are many English words

describing tricks, moves and equip-

ment. So, English lessons can reach

Ğǀ ĞƌǇŽŶĞD ĂƚŚƐŽƌŶĂƟǀ ĞůĂŶŐƵĂŐĞ

ƚĞĂĐŚĞƌƐƐƉĞŶĚĂůŽƚŽĨƟŵĞǁ ŝƚŚ

students but they don’t normally

discuss personal things with them

and, if they do, they are unable to

apply this in the lessons. English les-

sons, however, are more suited to

this.

“Another vital thing is tomake performance

meaningful for students,which can be done

through the context of atask. For this reason, tasksshould be correlated with

the students’ potentialknowledge and provide

them with relevantfeedback.”

13


Issue 63 13

A task can be ŵƵůƟŽƌĐƌŽƐƐ-subject

because, in real life, we rarely come

across isolated tasks – think of con-

ǀ ĞƌƐĂƟŽŶƐ ƐĞŶĚŝŶŐĞ-mails, working

on projects, travelling and so on.

We base our work on all our skills

and experience, so teachers should

ƚƌǇƚŽƐĞƚƚŚĞƐĂŵĞĐŽŶĚŝƟŽŶƐĨŽƌ

their students. Knowing what topics

are being discussed in history or

ŵĂƚŚƐĂƚƚŚĞƐĂŵĞƟŵĞĐĂŶŚĞůƉŝŶ

developing such tasks.

ůƚŚŽƵŐŚĂƐƐĞƐƐŵĞŶƚŝƐŽŌĞŶƐƚƌĞƐƐͲ

ĨƵůĨŽƌƐƚƵĚĞŶƚƐ ƐƚƌĞƐƐĚŽĞƐŶ ƚŽŌĞŶ

ŚĂǀ ĞŶĞŐĂƟǀ ĞƌĞƐƵůƚƐ ƐƚƌĞƐƐĨƵů

ƐŝƚƵĂƟŽŶĂůƐŽŵĂŬĞƐŝƚŵŽƌĞƌĞĂů

that is, closer to true foreign lan-

ŐƵĂŐĞĐŽŵŵƵŶŝĐĂƟŽŶ

By combining the above ideas, we

can turn assessment into ĂƵƚŚĞŶƟĐ

ĂƐƐĞƐƐŵĞŶƚ ƚŚĂƚŝƐ Ğǀ ĂůƵĂƟŽŶŝŶĂ

ƐŝƚƵĂƟŽŶǁ ŚĞƌĞĨŽƌĞŝŐŶůĂŶŐƵĂŐĞ

competence is applied when it is

ĞƐƐĞŶƟĂůĂŶĚŶĂƚƵƌĂů

There are several ways to make as-

ƐĞƐƐŵĞŶƚƉƌĂĐƟĐĂůKŶĞƚŚĂƚĐĂŶďĞ

controlled at all stages is a project.

A project calls for students to per-

form a variety of tasks and check

different goals. There can be several

stages, each with different aims and

ŽďũĞĐƟǀ ĞƐ

The first stage works as an introduc-

ƟŽŶƚŽƚŚĞƚŽƉŝĐt ƌŝƩ ĞŶĂŶĚĂƵĚŝŽ

tasks are given. If there is a need,

reading and listening skills can also

be tested here. The materials set

the context for all subsequent work.

At the second stage, students re-

ceive the project task, which usually

involves some research and produc-

Ɵǀ ĞƉĂƌƚƐ ƐƚƵĚǇƉůĂŶŝƐĞŝƚŚĞƌŐŝǀ Ͳ

en to the students or they develop

one themselves, with some points

ĂƐƋƵĞƐƟŽŶƐŽƌƐƚĂƚĞŵĞŶƚƐ dŚĞŶ

ƚŚĞǇĐƌĞĂƚĞĂƉƌĞƐĞŶƚĂƟŽŶ ƚƚŚĞ

ŶĞǆƚƐƚĂŐĞƚŚĞǇŐŝǀ ĞĂƉƌĞƐĞŶƚĂƟŽŶ

ĞƉĞŶĚŝŶŐŽŶƚŚĞŽďũĞĐƟǀ ĞƐ ƚŚĞ

teacher can add different tasks to

the project: group research, collec-

Ɵǀ ĞŽƌĂůƉƌĞƐĞŶƚĂƟŽŶ ǁ ƌŝƩ ĞŶƚĂƐŬ

ƉŽƐƚĞƌŽƌůĞƩ ĞƌdŚĞŵĂŝŶĂŝŵŽĨ

this stage is to see the language

competence the students have truly

acquired, and analyse their perfor-

mance. The teacher plays the role of

ŝŶƚĞƌůŽĐƵƚŽƌĂŶĚƐƟŵƵůĂƚĞƐĚŝƐĐƵƐͲ

ƐŝŽŶĂƌŽƵŶĚƚŚĞƉƌĞƐĞŶƚĂƟŽŶ ĂƐŬͲ

ŝŶŐĂĚĚŝƟŽŶĂůƋƵĞƐƟŽŶƐĂŶĚďƌŝŶŐͲ

ing up points on some details that

can be upgraded and improved. This

provides considerable speaking

ƉƌĂĐƟĐĞĂŶĚƉƌŽĚƵĐĞƐƌĞůĞǀ ĂŶƚŝŶͲ

ĨŽƌŵĂƟŽŶĂďŽƵƚƐŬŝůůƐůĞǀ ĞůƐ

The feedback stage is crucial for au-

ƚŚĞŶƟĐĂƐͲ

sessment.

The teach-

er can

ŝĚĞŶƟĨǇ

common

mistakes

more easily

when viewing students working to-

gether.

To sum up, I’d like to emphasise

that, once the teacher decides to

develop and assess competence

ĂĐƋƵŝƐŝƟŽŶŚŽůŝƐƟĐĂůůǇĂƵƚŚĞŶƟĐ

assessment features can be very

ŚĞůƉĨƵůƵƚŚĞŶƟĐĂƐƐĞƐƐŵĞŶƚĐĂŶ

be beneficial for both teachers and

students. The former obtain an op-

portunity to see to what extent

their students have really acquired

various competences, and to draw

conclusions about the efficacy of

ƚŚĞŝƌƚĞĂĐŚŝŶŐdŚĞůĂƩĞƌĐĂŶƚƌǇŽƵƚ

ƚŚĞŝƌƐŬŝůůƐƵŶĚĞƌŶĂƚƵƌĂůĐŽŶĚŝƟŽŶƐ

and understand their weaknesses

ĂŶĚĂďŝůŝƟĞƐďĞĨŽƌĞůĞĂǀ ŝŶŐƚŚĞƐĂĨĞͲ

ty of the classroom.

ǀ ĂůƵĂƟŽŶŝƐĂƚƌĞĂƐƵƌĞĐŚĞƐƚǁ ŚĞŶ

it comes to teaching languages, and

teachers should be familiar with it

ĂŶĚƵƐĞŝƚ D ƵůƟƉůĞĐŚŽŝĐĞƚĞƐƚƐ

ƐŚŽƵůĚďĞůĞŌĨŽƌŐĞŶĞƌĂůƚĞƐƟŶŐ

and other less fortunate subjects.

The EL teacher should try to make

ƚŚĞŵŽƐƚŽĨĞǀ ĂůƵĂƟŽŶ ĞŶƐƵƌŝŶŐ

Ğǀ ĞƌǇĂƐƐĞƐƐŵĞŶƚƐĞƐƐŝŽŶŝƐĂƉƌĂĐƟͲ

cal one.

14


Issue 63 14

What you can do for TEASIGIATEFL TEASIG members all have one thing in common: their interest in issues connected withƚĞƐƟŶŐĞǀ ĂůƵĂƟŽŶ ĂŶĚĂƐƐĞƐƐŵĞŶƚ ŽŵĞŵĞŵďĞƌƐĂƌĞĞǆƉĞƌƚƐŝŶƚŚĞĮ ĞůĚĂŶĚĐŽŶƚƌŝďƵƚĞƚŽƚŚĞ^/' ďǇƐŚĂƌŝŶŐƚŚĞŝƌĞǆƉĞƌƟƐĞǁ ŝƚŚŽƚŚĞƌŵĞŵďĞƌƐŝŶǀ ĂƌŝŽƵƐǁ ĂǇƐ ŚŽůĚŝŶŐǁ ĞďŝŶĂƌƐ ƐƉĞĂŬŝŶŐĂƚĐŽŶĨĞƌĞŶĐĞƐĂŶĚĞǀ ĞŶƚƐ ĂŶĚǁ ƌŝƟŶŐĨŽƌƚŚĞEĞǁ ƐůĞƩ ĞƌƐ/d&>ŝƐĂƚĞĂĐŚĞƌƐ ĂƐƐŽĐŝĂƟŽŶ ŵĂŶǇŵĞŵďĞƌƐĂƌĞƚĞĂĐŚĞƌƐ ƐŽŵĞũƵƐƚƐƚĂƌƟŶŐŽŶƚŚĞŝƌƚĞĂĐŚŝŶŐĐĂƌĞĞƌĂŶĚŽƚŚĞƌƐĂůŽŶŐǁ ĂǇŝŶƚŽŝƚ ŚŝŐŚŶƵŵďĞƌŽĨƚĞĂĐŚĞƌƐŚĂǀ ĞĂŐƌĞĂƚĚĞĂůŽĨĞǆƉĞƌŝĞŶĐĞŽĨƚĞƐƟŶŐǁ ŚŝůĞŶŽƚĐŽŶƐŝĚĞƌŝŶŐƚŚĞŵͲselves experts, whereas others are keen to get into what they feel is an important and dynamic areaof teaching EFL. Many teachers hope to learn something from experts and from other teachers, andƚŽŚĂǀ ĞƚŚĞŽƉƉŽƌƚƵŶŝƚǇƚŽĂƐŬƋƵĞƐƟŽŶƐĂŶĚƐŚĂƌĞƚŚĞŝƌŽǁ ŶŝĚĞĂƐ t ŚŝĐŚĞǀ ĞƌŐƌŽƵƉǇŽƵďĞůŽŶŐƚŽ ƚŚĞƌĞĂƌĞŵĂŶǇǁ ĂǇƐŝŶǁ ŚŝĐŚǇŽƵĐĂŶĐŽŶƚƌŝďƵƚĞƚŽd^/' ĂŶĚŽƵƌĞī ŽƌƚƐƚŽƉƌŽǀ ŝĚĞĞī ĞĐƟǀ ĞƐĞƌͲvices to all our members. Here are a few ideas.

E Ğǁ ƐůĞƩĞƌContribute

ďǇĂŶƐǁ ĞƌŝŶŐŵĞŵďĞƌƐƉŽƚůŝŐŚƚƋƵĞƐƟŽŶƐĂŶĚŐŝǀ ŝŶŐĂůůŵĞŵďĞƌƐĂŶŝĚĞĂŽĨǁ ŚŽĞůƐĞŝƐŝŶƚŚĞ /'

ďǇƐĞŶĚŝŶŐƵƐĂƐŚŽƌƚĂŶƐǁ ĞƌƚŽƚŚĞD ŽƐĂŝĐƋƵĞƐƟŽŶĨŽƌƚŚĞŶĞǆƚŝƐƐƵĞ

ďǇƌĞƉŽƌƟŶŐŽŶĂŶǇ;ŚŽǁ Ğǀ ĞƌƐŵĂůůͿƌĞƐĞĂƌĐŚŝŶƚŽĂƐƐĞƐƐŵĞŶƚŽƌƚĞƐƟŶŐǇŽƵŚĂǀ ĞĚŽŶĞ

ďǇƌĞƉŽƌƟŶŐŽŶĂŶĂƐƐĞƐƐŵĞŶƚŽƌƚĞƐƟŶŐĞǀ ĞŶƚŽƌĐŽŶĨĞƌĞŶĐĞǇŽƵŚĂǀ ĞĂƩ ĞŶĚĞĚHelp the Editors by

ĂƉƉƌŽĂĐŚŝŶŐƉŽƚĞŶƟĂůĐŽŶƚƌŝďƵƚŽƌƐ

ĂƉƉƌŽĂĐŚŝŶŐƉŽƚĞŶƟĂůĂĚǀ ĞƌƟƐĞƌƐ

proof-reading

Őŝǀ ŝŶŐĨĞĞĚďĂĐŬŽŶƚŚĞEĞǁ ƐůĞƩ Ğƌ;ǁ ŚĂƚǇŽƵůŝŬĞǁ ŚĂƚǇŽƵǁ ŽƵůĚůŝŬĞƚŽƐĞĞĂŶĚǁ ŚĂƚǇŽƵĚŽŶ ƚůŝŬĞ– ďƵƚƉůĞĂƐĞƉƌŽǀ ŝĚĞĂŶĂůƚĞƌŶĂƟǀ Ğǁ ŚĞƌĞƉŽƐƐŝďůĞ Ϳ

KƚŚĞƌƉƵďůŝĐĂƟŽŶƐ>ĞƚƵƐŬŶŽǁ ǁ ŚĂƚŽƚŚĞƌƉƵďůŝĐĂƟŽŶƐǁ ĞĐŽƵůĚŽī Ğƌ

WebsiteSend

ŝŶĨŽƌŵĂƟŽŶŽŶĞǆƚĞƌŶĂůĞǀ ĞŶƚƐ ƐĞŵŝŶĂƌƐ ǁ ŽƌŬƐŚŽƉƐ ĂŶĚĐŽŶĨĞƌĞŶĐĞƐ;ƐƚƌŝĐƚůǇĐŽŵŵĞƌĐŝĂůĂĐƟǀ ŝƟĞƐ including fee-ďĞĂƌŝŶŐĐŽƵƌƐĞƐ ǁ ŝůůŶŽƚďĞĂĚǀ ĞƌƟƐĞĚĨƌĞĞŽĨĐŚĂƌŐĞͿ ĚĞĐŝƐŝŽŶŽŶǁ ŚĞƚŚĞƌƚŽŝŶĐůƵĚĞĂŶǇĞǀ ĞŶƚŝƐĂƚƚŚĞĚŝƐĐƌĞƟŽŶŽĨƚŚĞ /' Ž-ordinator(s).

feedback on the website (what you like, would like to see, and don’t like)

WebinarsD ĂŬĞƐƵŐŐĞƐƟŽŶƐĨŽƌƐƉĞĂŬĞƌƐĂŶĚĨŽƌŵĂƚƐ

Events Help to organize an event in your area. Provide technical support with live-streaming and recording of sessions. Act as a helper at a TEASIG event.

Social media Encourage colleagues to join TEASIG social media groups. Start and contribute to discussions.

15


Issue 63 15

/ŶƚƌŽĚƵĐƟŽŶ

The teaching and assessment ofspoken English has arguably beenneglected by China’s exam-focusedĞĚƵĐĂƟŽŶĐƵůƚƵƌĞ&ŽƌŚŝŶĞƐĞƐƚƵͲdents, becoming confident users of ŶŐůŝƐŚŽī ĞƌƐŵǇƌŝĂĚŽƉƉŽƌƚƵŶŝƟĞƐ ďƵƚƐŝŶĐĞƐƵĐĐĞƐƐ ƐŽŽŌĞŶŵĞƌĞůǇequates to passing exams, many ofƚŚĞƐĞǇŽƵŶŐƉĞŽƉůĞŚĂǀ ĞůŝƩ ůĞĐŽŶͲĐĞƉƟŽŶŽĨůŝĨĞůŽŶŐůĞĂƌŶŝŶŐ/Ɛƚƌŝǀ ĞĚƚŽĐŚĂŶŐĞƚŚŝƐĂƫ ƚƵĚĞǁ ŝƚŚŵǇƐƚƵͲdents by designing a speaking testwhich aimed to be both useful andmeaningful: a spontaneous groupĐŽŶǀ ĞƌƐĂƟŽŶĂĐƟǀ ŝƚǇdŚŝƐĂƌƟĐůĞbegins with a few background de-tails and then explains the test’sdesign and marking procedures.

The learners and the course

My class, at a public university inInner Mongolia, China, consisted ofĮ ŌĞĞŶD ĂŶĚĂƌŝŶ>ϭůĞĂƌŶĞƌƐǁ ŚŽĂůůstruggled with oral English to somedegree. As English majors, theircompulsory modules included gram-ŵĂƌůŝƐƚĞŶŝŶŐƌĞĂĚŝŶŐǁ ƌŝƟŶŐĂŶĚƚƌĂŶƐůĂƟŽŶ ŵŽƐƚůǇƚĂƵŐŚƚďǇŶĂƟǀ ĞŚŝŶĞƐĞƚĞĂĐŚĞƌƐŝŶƉƌĞƉĂƌĂƟŽŶĨŽƌŶĂƟŽŶĂůĞǆĂŵƐůŝŬĞdD ;dĞƐƚĨŽƌEnglish Majors) and CET (CollegeEnglish Test). Since oral fluency is not integral to these exams, speak-ŝŶŐĐůĂƐƐĞƐĂƌĞŶŽƌŵĂůůǇůĞŌƚŽĨŽƌͲĞŝŐŶƚĞĂĐŚĞƌƐ ǁ ŚŽƐĞŝŶƚĞƌĂĐƟǀ ĞƚĞĂĐŚŝŶŐŵĞƚŚŽĚƐŽŌĞŶĐŽŶƚƌĂĚŝĐƚƚŚĞƚƌĂĚŝƟŽŶĂůƚĞĂĐŚĞƌ-centred cur-riculum. Because of these factors,most of my learners were anxiousabout speaking spontaneously inĐůĂƐƐĂŶĚŽŌĞŶŵĞŵŽƌŝƐĞĚĂŶĚƌĞͲĐŝƚĞĚƚĞǆƚƐĚƵƌŝŶŐƉƌĞƐĞŶƚĂƟŽŶƐŽƌ

ŽƚŚĞƌĂĐƟǀ ŝƟĞƐ ŵĂŬŝŶŐŝƚĐŚĂůůĞŶŐͲing to observe or obtain samples of‘natural’ speech to monitor, andƵůƟŵĂƚĞůǇĂƐƐĞƐƐ ƚŚĞŝƌƉƌŽŐƌĞƐƐ

Developing the confidence, knowledge and skills required forengaging in informal discussion wasƚŚĞĐĞŶƚƌĂůůĞĂƌŶŝŶŐŽďũĞĐƟǀ ĞŽĨŽƵƌ18-week course. Our weekly lessonsfocused on various themes – ŽŌĞŶsuggested by the learners – and in-cluded related vocabulary andŐƌĂŵŵĂƟĐĂůƐƚƌƵĐƚƵƌĞƐǁ ŝƚŚƌĞŐƵůĂƌƉĂŝƌĂŶĚŐƌŽƵƉĚŝƐĐƵƐƐŝŽŶĂĐƟǀ ŝƟĞƐƚŽŵĂǆŝŵŝƐĞƐƉĞĂŬŝŶŐŽƉƉŽƌƚƵŶŝƟĞƐas well as build knowledge and skillsƚŚƌŽƵŐŚĂĐƟǀ ĞƉƌĂĐƟĐĞdŽĨŽƐƚĞƌ>ϮĐŽŶĮ ĚĞŶĐĞƚŚĞĨŽĐƵƐǁ ĂƐƐŚŝŌĞĚfrom accuracy to fluency and I en-couraged the learners to speak asmuch as possible without worryingabout mistakes. They had good re-ĐĞƉƟǀ ĞŬŶŽǁ ůĞĚŐĞŽĨŐƌĂŵŵĂƌƐŽ/ŬŶĞǁ ǁ ŝƚŚƉƌĂĐƟĐĞƚŚĞǇĐŽƵůĚŐƌĂĚͲually begin to noƟce and correcttheir errors.

Test design

ŽŶĚƵĐƟŶŐĂŶĞŶĚ-of-term achieve-ment exam was a course require-ment, but I was given the freedomto design, administer and mark theƚĞƐƚ ĐƌƵĐŝĂůĐŽŶƐŝĚĞƌĂƟŽŶĨŽƌŶŽŶ-ƚƌĂĚŝƟŽŶĂůĂƐƐĞƐƐŵĞŶƚŝƐƚŚĂƚƚĞƐƚƐshould be “integrated with the goalsof the curriculum and … have a con-ƐƚƌƵĐƟǀ ĞƌĞůĂƟŽŶƐŚŝƉǁ ŝƚŚƚĞĂĐŚŝŶŐand learning” (McNamara, 2000,Ɖ ϳ Ϳ ĐŽŶǀ ĞƌƐĂƟŽŶ ƚĞƐƚĂĐƟǀ ŝƚǇclearly reflected the course objec-Ɵǀ ĞĂŶĚƚŚĞĐŽŵŵƵŶŝĐĂƟǀ ĞŇƵĞŶĐǇ-based classroom content, thusfirmly linking the test to the learning

Maximising collaboration in oral English testing:an example from China

Susanna Wickes

began teaching in her hometown of Edinburgh in 2009.Since then she has taught inAustralia, India, Ireland and -most extensively - China. Dur-ing her years in this countryshe has developed researchinterests in assessment, learn-er autonomy, and collabora-Ɵǀ ĞůĞĂƌŶŝŶŐƉĂƌƟĐƵůĂƌůǇŝŶthe university context. Shecurrently teaches EAP at thehŶŝǀ ĞƌƐŝƚǇŽĨE Žƫ ŶŐŚĂŵNingbo China.

[email protected]

This paper is based on aƉƌĞƐĞŶƚĂƟŽŶŐŝǀ ĞŶĂƚƚŚĞTEASIG Day at the IATEFLConference in Glasgow inApril 2017.

16


Issue 63 16

goal. The test results then enabledthe learners to gauge their progress,diagnose their needs, and set newgoals.

The individual competences re-ƋƵŝƌĞĚĨŽƌĐŽŶǀ ĞƌƐĂƟŽŶĂŶĚŝŶĨŽƌͲmal discussion were adapted fromthe CEFR (Council of Europe, 2001)and are shown below with a de-ƐĐƌŝƉƟŽŶŽĨƚŚĞĂďŝůŝƚǇůĞǀ Ğů/ĞǆͲpected my learners to reach in eachcategory. These competences wereƚĂƵŐŚƚĂŶĚƉƌĂĐƟƐĞĚƚŚƌŽƵŐŚŽƵƚƚŚĞterm and provided the basis for themarking rubrics.

>ŝŶŐƵŝƐƟĐĐŽŵƉĞƚĞŶĐĞƐ

Phonological

Lexical

' ƌĂŵŵĂƟĐĂů

^ĞŵĂŶƟĐ

The learner can use ap-propriate and intelligiblewords, expressions andstructures, with clear, intel-ůŝŐŝďůĞƉƌŽŶƵŶĐŝĂƟŽŶ ƚŽexpress his/her ideas.

WƌĂŐŵĂƟĐĐŽŵƉĞƚĞŶĐĞƐ

Discourse competence

&ƵŶĐƟŽŶĂůĐŽŵƉĞƚĞŶĐĞ

The learner can use ap-propriate techniques to ini-ƟĂƚĞĞŶŐĂŐĞŝŶĂŶĚŵĂŝŶͲƚĂŝŶĐŽŶǀ ĞƌƐĂƟŽŶ ĐĂŶĂƐŬƋƵĞƐƟŽŶƐ ĂŐƌĞĞĚŝƐĂŐƌĞĞĞƚĐ ĐĂŶŐŝǀ ĞĚĞƐĐƌŝƉƟŽŶƐ ĞǆƉůĂŶĂƟŽŶƐ ĞƚĐǁ ŚĞƌĞappropriate, and can per-form all of these in a confi-dent manner without exces-Ɛŝǀ ĞƉĂƵƐŝŶŐŽƌŚĞƐŝƚĂƟŽŶ

Test method and content

ůƚŚŽƵŐŚĐŽŶǀ ĞƌƐĂƟŽŶ-style testscan take place between two people– a test taker and an assessor/interlocutor or two test takers – Iorganised my learners into groupsof three as talking in groups can re-duce anxiety (He & Dai, 2006). PeerŝŶƚĞƌĂĐƟŽŶĂůƐŽƉƌŽŵŽƚĞƐŵŽƌĞƌĞͲĂůŝƐƟĐĐŽŵŵƵŶŝĐĂƟŽŶƚŚĂŶĂƐƐĞƐƐŽƌ-ĞǆĂŵŝŶĞĞĐŽŶǀ ĞƌƐĂƟŽŶĂƐůĞĂƌŶĞƌƐare in an “equal power posi-ƟŽŶ ;>ƵŽŵĂϮϬϬϰ Ɖϭϴϳ ͿĂŶĚĐĂŶĚĞŵŽŶƐƚƌĂƚĞŵŽƌĞŝŶƚĞƌĂĐƟǀ ĞĨƵŶĐͲƟŽŶƐƚŚĂŶƐŝŵƉůǇĂŶƐǁ ĞƌŝŶŐĂŶŝŶͲƚĞƌůŽĐƵƚŽƌƐƋƵĞƐƟŽŶƐ dŚŝƐĞŶĂďůĞƐthe assessor to observe, rather than

ƉĂƌƟĐŝƉĂƚĞŝŶƚŚĞĂĐƟǀ ŝƚǇĂŶĚƚŚĞƌĞͲĨŽƌĞƉĂǇŵŽƌĞĂƩĞŶƟŽŶ ĚĚŝƟŽŶĂůͲůǇŐƌŽƵƉĂƐƐĞƐƐŵĞŶƚŝƐƉƌĂĐƟĐĂůĂŶĚĞĸ ĐŝĞŶƚ ƚŚĞĮ ŌĞĞŶƚĞƐƚƚĂŬĞƌƐworked in five groups, allowing all the tests to be completed duringone two-hour lesson.

Open-ĞŶĚĞĚĐŽŶǀ ĞƌƐĂƟŽŶĐŽŶƚĞŶƚwas prompted using topic cards.Before the test week, six broadthemes covered in the course wereĐŚŽƐĞŶďǇƐƚƵĚĞŶƚǀ ŽƚĞĂŶĚǁ ƌŝƩ ĞŶon the backs of six cards. During thetest, one card was randomly select-ĞĚƚŽŝŶƐƉŝƌĞĂƐŚŽƌƚĐŽŶǀ ĞƌƐĂƟŽŶ The topics – personality, technology,

ƌĞůĂƟŽŶƐŚŝƉƐ ĞŶǀ ŝƌŽŶŵĞŶƚ ƚƌĂǀ Ğůand ĞĚƵĐĂƟŽŶ – were all familiar tothe learners and were consideredequally ‘difficult’ as they could all be discussed at any level from basicand concrete to more abstract andcomplex. Knowing the six topics inadvance allowed learners to prac-ƟƐĞĂŶĚƌĞǀ ŝƐĞƌĞůĞǀ ĂŶƚǀ ŽĐĂďƵůĂƌǇand grammar, thus reducing nervesand maximising fairness, but notƌĞǀ ĞĂůŝŶŐƚŚĞƐƉĞĐŝĮ ĐƚŽƉŝĐƵŶƟůƚŚĞtest ensured that learners could not ĐŚĞĂƚ ďǇƐĐƌŝƉƟŶŐĂŶĚŵĞŵŽƌŝƐŝŶŐƚŚĞŝƌĐŽŶǀ ĞƌƐĂƟŽŶƐ

dĞƐƚĂĚŵŝŶŝƐƚƌĂƟŽŶ

ŌĞƌĞǆƉůĂŝŶŝŶŐĂŶĚƉƌĂĐƟƐŝŶŐthe assessment format, the testwas administered during classƟŵĞŝŶƚŚĞƐĞĐŽŶĚ-last week ofterm, leaving the final week free for a peer/self-Ğǀ ĂůƵĂƟŽŶƐĞƐͲƐŝŽŶ /ŝŶŝƟĂƚĞĚĂƐŚŽƌƚ ƵŶĂƐͲsessed ‘warm up’ to help the testtakers relax, then a group mem-

ber was asked to randomly selectone of the six cards, which providedƚŚĞƚŽƉŝĐĨŽƌƚŚĞĐŽŶǀ ĞƌƐĂƟŽŶ /ƐƚĂƌƚĞĚƚŚĞƌĞĐŽƌĚĞƌĂŶĚƐĞƚĂƟŵĞƌǁ ŚŝĐŚďĞĞƉĞĚĂŌĞƌƐĞǀ ĞŶŵŝŶƵƚĞƐto let the group know they hadreached the minimum limit for com-ƉůĞƟŶŐƚŚĞƚĂƐŬdŚŝƐŵĞƚŚŽĚĂůͲlowed the learners to end their dis-cussion when and how they wished,thus producing a more representa-Ɵǀ ĞƵŶĐŽŶƚƌŝǀ ĞĚĐŽŶǀ ĞƌƐĂƟŽŶƐĂŵͲƉůĞdŚĞƟŵĞƌǁ ĂƐƐĞƚƚŽďĞĞƉĂŐĂŝŶĂŌĞƌƚĞŶŵŝŶƵƚĞƐ– the upper limit –to tell the group to round things off ŝĨƚŚĞǇǁ ĞƌĞƐƟůůƚĂůŬŝŶŐE ŽƚĞƐǁ ĞƌĞnot allowed before or during theĐŽŶǀ ĞƌƐĂƟŽŶĂŶĚƚĞƐƚƚĂŬĞƌƐǁ ĞƌĞ

“Although conversation-styletests can take placebetween two people

I organised my learners intogroups of three as talking in

groups can reduce anxiety.”

17


Issue 63 17

ŶŽƚŐŝǀ ĞŶƟŵĞƚŽƉƌĞƉĂƌĞdŚŝƐǁ ĂƐa deliberate decision to maximiseĂƵƚŚĞŶƟĐŝƚǇĂƐƌĞĂůĐŽŶǀ ĞƌƐĂƟŽŶƐare spontaneous. I only intervenedŝĨƚŚĞĐŽŶǀ ĞƌƐĂƟŽŶŚĂĚĐŽŵƉůĞƚĞůǇhalted or if the learners directlyĂƐŬĞĚŵĞĂƋƵĞƐƟŽŶ ǁ ŚŝĐŚƚŚĞǇcould choose to do.

Teacher marking

/ŶĂƵƚŚĞŶƟĐƚĞƐƚƐŝƚŝƐǀ ŝƚĂůƚŽŝŶǀ Žůǀ Ğlearners in the assessment processas it enables them to developawareness and take responsibilityfor their own learning (O’Malley &Pierce, 1996), however peer andself-ĂƐƐĞƐƐŵĞŶƚĐĂŶŽŶůǇƌĞĂůŝƐƟĐĂůͲly supplement the expert evalua-ƟŽŶŽĨƚŚĞƚĞĂĐŚĞƌ;>ƵŽŵĂϮϬϬϰͿ For these reasons, I made markingĂĐŽůůĂďŽƌĂƟǀ ĞĞŶĚĞĂǀ ŽƵƌǁ ŝƚŚĂŶĂŶĂůǇƟĐƌĂƟŶŐƐĐĂůĞĨŽƌƚĞĂĐŚĞƌĂƐͲƐĞƐƐŵĞŶƚĂŶĚĂƐĞƉĂƌĂƚĞŚŽůŝƐƟĐscale for peer and self-assessment,ƚŚĞůĂƩ ĞƌĚĞǀ ŝƐĞĚďǇƚŚĞůĞĂƌŶĞƌƐ dŚĞƚĞĂĐŚĞƌƐƐĐĂůĞǁ ĂƐǁ ƌŝƩ ĞŶƚŽrate and score individual test takers’ƉĞƌĨŽƌŵĂŶĐĞƐŝŶƚŚĞĐŽŶǀ ĞƌƐĂƟŽŶƐ and was adapted from the commu-ŶŝĐĂƟǀ ĞůĂŶŐƵĂŐĞĐŽŵƉĞƚĞŶĐĞƐŵĞŶƟŽŶĞĚĞĂƌůŝĞƌ

1. >ŝŶŐƵŝƐƟĐĐŽŵƉƌĞŚĞŶƐŝďŝůŝƚǇ

2. Phonological comprehensibility

3. Fluency

4. ŽŶǀ ĞƌƐĂƟŽŶĂůŝŶƚĞƌĂĐƟŽŶ

5. Evidence of learning

>ŝŶŐƵŝƐƟĐĐŽŵƉƌĞŚĞŶƐŝďŝůŝƚǇǁ ĂƐincluded to discourage test takersfrom overly focusing on accuracyand thereby threatening fluency. It was defined as being able to use

appropriate and intelligible wordsand construct intelligible phrasesand sentences. Learners weremarked down if their errors imped-ed understanding, but mistakes andslips were disregarded. Pronuncia-ƟŽŶǁ ĂƐĂůƐŽĚĞĮ ŶĞĚŝŶƚĞƌŵƐŽĨintelligibility, though a high score inthis category also involved goodĐŽŶƚƌŽůŽĨŝŶƚŽŶĂƟŽŶĂŶĚƐǇůůĂďůĞstress. Fluency was defined accord-ing to the learning goal of speaking

spontaneously and confidently with-out unnatural pauses and excessiveŚĞƐŝƚĂƟŽŶ /ŶƚĞƌĂĐͲƟŽŶŝŶĐůƵĚĞĚƚƵƌŶͲtaking, co-ŽƉĞƌĂƟŶŐĂŶĚƉĂƌƟĐŝƉĂƟŽŶ Finally, ‘evidence oflearning’ was includ-ed to assess learn-ers’ use and under-standing of taughtcontent, rated ac-cording to its fre-quency, accuracyand appropriate us-age.

I adapted five de-scriptors from theCEFR and wrotethem as five-point ƐĐĂůĞƐŝŶĂƐƉŽƐŝƟǀ Ğwording as possible.

During the test, I simplified this into a basic 5x5 grid so I could makeƋƵŝĐŬŶŽƚĞƐĂŌĞƌĞĂĐŚĐŽŶǀ ĞƌƐĂƟŽŶand then use these with the full ru-bric when listening to the record-ings.

Peer and self-assessment

Another rubric was created for peerand self-Ğǀ ĂůƵĂƟŽŶ ĂŶĚƚŽŐŝǀ ĞĂgroup score. This five-ƉŽŝŶƚŚŽůŝƐƟĐscale made for an efficient, ƐƚƌĂŝŐŚƞŽƌǁ ĂƌĚŵĂƌŬŝŶŐƉƌŽĐĞƐƐthat promoted discussion and nego-ƟĂƟŽŶĂŵŽŶŐƐƚůĞĂƌŶĞƌƐ dŽŵĂǆͲimise learner involvement in thisprocess, the class was responsibleĨŽƌĐƌĞĂƟŶŐƚŚĞƐĐĂůĞĚƵƌŝŶŐŽŶĞŽĨtheir lessons. In small groups, theywere asked write a short descrip-ƟŽŶŽĨǁ ŚĂƚƚŚĞǇƚŚŽƵŐŚƚĐŽŶƐƟͲtuted an ‘excellent’ and a ‘not so

good’ performance, and eventually,ĂŌĞƌƐŽŵĞƌĞƉŽƌƟŶŐĂŶĚĐŽŵƉĂƌͲing, we wrote a five-point scale that

“To maximise learner

involvement in this process,

the class was responsible

for creating the scale

during one of their lessons.”

Teacher’s marking rubric (adapted from CEFR)

18


Issue 63 18

everyone agreed on.

ǁ ĞĞŬĂŌĞƌƚŚĞƚĞƐƚ ƚŚĞůĞĂƌŶĞƌƐreturned to their groups and Iplayed the recordings. Each groupagreed on a number from one tofive to score each recording, includ-ing their own, and wrote some con-ƐƚƌƵĐƟǀ ĞĐŽŵŵĞŶƚƐŽŶĂŶĂŶŽŶǇͲŵŽƵƐĞǀ ĂůƵĂƟŽŶƐŚĞĞƚ /ĨĂŶƵŵďĞƌcould not be agreed on, groupmembers could award a half marksuch as 3.5. I recorded the scores

and calculated an average for eachgroup, adding it to all the membersof that group on top of their individ-ual scores. At the end of class theĞǀ ĂůƵĂƟŽŶƐŚĞĞƚƐǁ ĞƌĞĚŝƐƚƌŝďƵƚĞĚto the learners to be read and dis-cussed.

Group marks encourage learners towork together and take joint re-sponsibility for their performancewhile individual marks make learn-ers accountable for their own con-ƚƌŝďƵƟŽŶƐ;K D ĂůůĞǇΘWŝĞƌĐĞϭϵϵϲͿ dŚƵƐ ĂůůŽĐĂƟŶŐďŽƚŚŵĂǆŝŵŝƐĞƐƚŚĞďĞŶĞĮ ƚƐŽĨĐŽůůĂďŽƌĂƟǀ ĞůĞĂƌŶŝŶŐyet remains fair for everyone.

A note on validity

Success in the test provided me and

learners with evidence that thelearning goal had been achieved;the students were able to haveƐƉŽŶƚĂŶĞŽƵƐŶŐůŝƐŚĐŽŶǀ ĞƌƐĂƟŽŶƐ However, the nature of open-endedŐƌŽƵƉĐŽŶǀ ĞƌƐĂƟŽŶƚĞƐƚƐƉŽƐĞƐsome validity issues. Assessinglearners in groups introduces nu-merous variables, making it difficult to isolate individual performances.t ŚŝůĞƚŚŝƐŝƐƉƌŽďůĞŵĂƟĐŝƚŵƵƐƚďĞƌĞŵĞŵďĞƌĞĚƚŚĂƚĐŽŶǀ ĞƌƐĂƟŽŶŝƐ

inherent-ly collab-oraƟveand con-textbound,and soŝƐŽůĂƟŶŐspeechand testitems

would invalidate the whole test.

Another issue is that learners inevi-tably act differently when being as-sessed (Luoma, 2004), most likelydue to test anxiety and the desire todemonstrate their best perfor-mance (He & Dai, 2006). Of courseneither of these would be present in ƌĞĂůǁ ŽƌůĚ ĐŽŶǀ ĞƌƐĂƟŽŶƐ dŽŵŝŶŝͲmise such threats I did everythingpossible to ensure my learners feltĂƚĞĂƐĞŝŶĐůƵĚŝŶŐŐŝǀ ŝŶŐƚŚĞŵƟŵĞƚŽƉƌĞƉĂƌĞĂŶĚƉƌĂĐƟĐĞĂŶĚĂůůŽǁ Ͳing them to choose their owngroups.

Conclusion

dŚŝƐĂƌƟĐůĞŚĂƐŽƵƚůŝŶĞĚĂƐŝŵƉůĞǇĞƚŝŶŶŽǀ ĂƟǀ ĞƐƉĞĂŬŝŶŐƚĞƐƚ ĚĞƐŝŐŶĞĚto help a class of Chinese university

students develop the skills,knowledge and confidence to en-gage in English language conversa-ƟŽŶƐ dŚĞĐůĂƐƐƌŽŽŵĐŽŶƚĞŶƚƚŚĂƚprovided the basis of the test facili-tated a meaningful and valuablelearning experience, and the collab-oraƟve evaluaƟon process promot-ed inter-peer dialogue and encour-aged learners to take control oftheir learning. Though group testsintroduce inevitable threats to valid-ity I took acƟon to minimise these,ĐƌĞĂƟŶŐĂƚĞƐƚƚŚĂƚ/ĐŽŶƚĞŶĚŝƐĂƵͲƚŚĞŶƟĐĨĂŝƌĂŶĚƵƐĞĨƵů

References

Council of Europe. 2001. Common Euro-pean Framework of Reference for Lan-guages: learning, teaching, assessment.Cambridge: Cambridge University Press.

He, Lianzhen & Dai, Ying. 2006. A cor-pus-ďĂƐĞĚŝŶǀ ĞƐƟŐĂƟŽŶŝŶƚŽƚŚĞǀ ĂůŝĚŝƚǇof the CET-SET group discus-sion. >ĂŶŐƵĂŐĞdĞƐƟŶŐ, 23(3): 370-401.

Luoma, Sari. 2004. Assessing speaking.Cambridge: Cambridge University Press.

McNamara, Tim. 2000. LanguageƚĞƐƟŶŐ. Oxford: Oxford University Press.

O’Malley, Michael & Pierce, Lorraine V.1996. ƵƚŚĞŶƟĐĂƐƐĞƐƐŵĞŶƚĨŽƌŶŐůŝƐŚůĂŶŐƵĂŐĞůĞĂƌŶĞƌƐ ƉƌĂĐƟĐĂůĂƉƉƌŽĂĐŚĞƐfor teachers. New York: Longman.

Peer and self-assessment rubric

19


Issue 63 19

/ŶĂƌĞĐĞŶƚũŽƵƌŶĂůĂƌƟĐůĞ/ƉƌĞƐĞŶƚͲ

ĞĚŵǇŝŶŝƟĂůƌĞƐĞĂƌĐŚŝŶƚŽƐƚƵĚĞŶƚƐ

openness to a new approach to giv-

ŝŶŐĨĞĞĚďĂĐŬŽŶǁ ƌŝƩ ĞŶǁ ŽƌŬ–

Learner-Driven Feedback (LDF).

Here is the reference for the full

ĂƌƟĐůĞ;Ăǀ ĂŝůĂďůĞŽƉĞŶůǇĂƐĚŝƚŽƌƐ

Choice on the ELTJ website): Maas,

ZĞĐĞƉƟǀ ŝƚǇƚŽ>ĞĂƌŶĞƌ-Driven

Feedback in EAP’, ELT Journal, 71/2

(2017), pp. 127-140.

The LDF approach understands that

ůĞĂƌŶĞƌƐŽŌĞŶĨĞĞůŚĂƉƉŝĞƌƌĞĐĞŝǀ ŝŶŐ

feedback from a teacher than

through peer review or self-ĞĚŝƟŶŐ

In LDF, learners ‘drive’ feedback by

ĂƐŬŝŶŐƋƵĞƐƟŽŶƐĂďŽƵƚƚŚĞŝƌǁ ŽƌŬ

and then re-ĚƌĂŌďĂƐĞĚŽŶƚŚŝƐ

feedback. The teacher gives the

feedback, but the learners decide

how and on what they receive com-

ments: they can choose between

various formats (e.g. hand-ǁ ƌŝƩ ĞŶ

email, audio recording), and are re-

ƋƵŝƌĞĚƚŽƉŽƐĞƋƵĞƐƟŽŶƐƚŽǁ ŚŝĐŚ

the teacher responds (e.g. about

grammar, vocabulary/register, ref-

erencing, text structure).

Some Background

I devised this approach by combin-

ing insights from other fairly recent

studies on giving feedback on L2

ǁ ƌŝƟŶŐƉĂƌƟĐƵůĂƌůǇŝŶWĐŽŶƚĞǆƚƐ

I looked into two key areas:

Making feedback more dialogic

Using technology to deliver

feedback

With regard to involving students in

a feedback dialogue, Bloxham and

ĂŵƉďĞůů;ϮϬϭϬͿƚƌŝĂůůĞĚ/ŶƚĞƌĂĐƟǀ Ğ

Coversheets’, which students use to

ƉŽƐĞƋƵĞƐƟŽŶƐĂďŽƵƚƚŚĞŝƌĞƐƐĂǇƐ

ǁ ŚĞŶƐƵďŵŝƫ ŶŐĚƌĂŌƐ ůŽǆŚĂŵ

and Campbell found that most stu-

dents in their study were grateful to

receive feedback that was more in-

dividualised than usual, which then

ĂůƐŽŵŽƟǀ ĂƚĞĚƚŚĞŵƚŽĞǀ ĂůƵĂƚĞ

their own work more thoroughly.

Moreover, the teachers involved

reported being able to provide feed-

ďĂĐŬŽŶĚƌĂŌƐŵŽƌĞƋƵŝĐŬůǇďǇŽŶůǇ

ĐŽŵŵĞŶƟŶŐŽŶĂƌĞĂƐŚŝŐŚůŝŐŚƚĞĚŝŶ

ƐƚƵĚĞŶƚƐ ƋƵĞƐƟŽŶƐ

Campbell and Schumm-Fauster

(2013) also required their EAP stu-

ĚĞŶƚƐƚŽƉŽƐĞĨĞĞĚďĂĐŬƋƵĞƐƟŽŶƐ

ŽŶƚŚĞŝƌǁ ŽƌŬǁ ŚĞŶƐƵďŵŝƫ ŶŐĞƐͲ

ƐĂǇĚƌĂŌƐ , ĞƌĞƐƚƵĚĞŶƚƐŶŽƚĞĚ

ƚŚĞŝƌƋƵĞƐƟŽŶƐŝŶĨŽŽƚŶŽƚĞƐŽƌŝŶ

the text’s margins. The students

were surveyed and reported finding

ƚŚĞĨĞĞĚďĂĐŬƉĞƌƐŽŶĂůĂŶĚŵŽƟͲ

ǀ ĂƟŶŐĂŶĚĞī ĞĐƟǀ ĞŝŶŵĞĞƟŶŐ

their individual needs regarding aca-

ĚĞŵŝĐǁ ƌŝƟŶŐ

Studies into using technology to de-

liver feedback have explored, for

example, using audio recordings or

emails. Johanson (1999) lists some

advantages of recording audio feed-

Clare Maas

holds post-ŐƌĂĚƵĂƚĞƋƵĂůŝĮ ĐĂƟŽŶƐfrom the University of Wales andTrinity College London. BeforeŵŽǀ ŝŶŐŝŶƚŽƚĞƌƟĂƌǇĞĚƵĐĂƟŽŶ she taught English at Germangrammar schools, and English forSpecific Purposes at several lan-guage academies in the UK andGermany. Her professional inter-ests include EAP, materials devel-opment, and CPD for teachers.Her most recent research hasďĞĞŶŝŶƚŽĨĞĞĚďĂĐŬŽŶ>Ϯǁ ƌŝƟŶŐespecially in EAP and higher edu-ĐĂƟŽŶ ŚĞĂůƐŽďůŽŐƐĂƚClaresELTCompendi-um.wordpress.com and is a mem-ber of the team behind EL-TResearchBites.com.

[email protected]

.

Receptivity to Learner-Driven Feedback

20


Issue 63 20

back for ESL students, which include

ŝƚƐƉŽƐƐŝďůǇƐĂǀ ŝŶŐƚĞĂĐŚĞƌƐƟŵĞ

making comments clearer through

ŝŶƚŽŶĂƟŽŶ ĂŶĚĨĞĞůŝŶŐŵŽƌĞƉĞƌͲ

sonal to students. These ideas are

supported by studies outside ELT,

such as Brearley and Cullen (2012),

who found three minutes of audio

recording could include ~500 words

of feedback and did not take more

ƟŵĞƚŚĂŶŵĂƌŬŝŶŐĞƐƐĂǇƐďǇŚĂŶĚ

ůƐŽŝŶǀ ĞƐƟŐĂƟŶŐĚŝŐŝƚĂůĨĞĞĚďĂĐŬ

Farshi and Safa (2015) compared

grades of EFL students who re-

ceived hand-ǁ ƌŝƩ ĞŶ ĞŵĂŝůĞĚ Žƌ

ŶŽĐŽƌƌĞĐƟǀ ĞĨĞĞĚďĂĐŬŽŶǁ ƌŝƩ ĞŶ

tasks for one term, and found

that emailed feedback led to sig-

nificantly greater improvement

on post-test grades than hand-

ǁ ƌŝƩ ĞŶĨĞĞĚďĂĐŬ

My Research

To put this summary in context,

readers need to know that I used

>&ŽŶƚŚƌĞĞĚƌĂŌƐŽĨĂŶĞƐƐĂǇ

that my undergraduate EAP stu-

dents (B2 level on CEFR) wrote dur-

ing one semester at a German uni-

versity. The students could:

choose between various (not

mutually exclusive!) modes of

feedback: in-ƚĞǆƚĐŽƌƌĞĐƟŽŶƐ

ĐŽƌƌĞĐƟŽŶƐǇŵďŽůƐ ŚĂŶĚǁ ƌŝƩ ĞŶ

feedback, email, audio recording,

or face-to-ĨĂĐĞĐŽŶƐƵůƚĂƟŽŶ

ĂƐŬƋƵĞƐƟŽŶƐĂďŽƵƚĂŶǇaspect

ŽĨƚŚĞŝƌǁ ƌŝƟŶŐĞ ŐƚĞǆƚƐƚƌƵĐͲ

ture, referencing, vocabulary,

grammar, etc.

pose both specific and general

ƋƵĞƐƟŽŶƐ

ŝŶĐůƵĚĞƚŚĞŝƌƋƵĞƐƟŽŶƐĂƐĨŽŽƚͲ

notes or in margins, or at the end

of the text.

At the end of semester, I surveyed

the 40 students to discover their

Ăƫ ƚƵĚĞƐƚŽǁ ĂƌĚƐƚŚĞ>&ĂƉƉƌŽĂĐŚ

dŚĞƋƵĞƐƟŽŶŶĂŝƌĞŝŶĐůƵĚĞĚƋƵĞƐͲ

ƟŽŶƐŽŶƐƚƵĚĞŶƚƐ ƉĞƌĐĞƉƟŽŶŽĨƚŚĞ

usability of various delivery modes,

ƚŚĞƉĞƌĐĞŝǀ ĞĚĞī ĞĐƟǀ ĞŶĞƐƐŽĨ>&

for improving their language accura-

cy and academic skills related to

essays, and any problems they ex-

perienced with LDF.

Some Findings

The delivery modes most students

requested were audio recording (by

67% of students) and email (60%).

dŚĞĮ ŐƵƌĞƐĨŽƌŚĂŶĚǁ ƌŝƩ ĞŶĐŽŵͲ

ments (13%) and in-ƚĞǆƚĐŽƌƌĞĐƟŽŶƐ

(20%) were much lower, which

seems to show that students are

happy to move away from

ƚƌĂĚŝƟŽŶĂůŚĂŶĚ-ǁ ƌŝƩ ĞŶĨĞĞĚďĂĐŬ

Students’ comments to open-ended

ƐƵƌǀ ĞǇƋƵĞƐƟŽŶƐŚŝŐŚůŝŐŚƚĞĚƐŽŵĞ

reasons why audio recordings and

email were so popular, for exam-

ƉůĞŵĞŶƟŽŶŝŶŐƚŚĂƚĨĞĞĚďĂĐŬǁ ĂƐ

ŵŽƌĞĚĞƚĂŝůĞĚĂŶĚƟŵĞůǇĂŶĚĨĞůƚ

more individualized than the feed-

back they were used to, and that,

ŝŶĂƵĚŝŽƌĞĐŽƌĚŝŶŐƐ ŝŶƚŽŶĂƟŽŶ

ŚĞůƉĞĚƚŚĞŵƚŽŝĚĞŶƟĨǇƉŽƐŝƟǀ Ğ

ĂŶĚŶĞŐĂƟǀ ĞĐŽŵŵĞŶƚƐ

Y ƵĞƐƟŽŶƐĂůƐŽĂƐŬĞĚǁ ŚŝĐŚĂƐƉĞĐƚƐ

of general language accuracy stu-

dents felt had significantly or

somewhat improved by working

with LDF. High numbers of students

felt they had improved their use of

ƚƌĂŶƐŝƟŽŶƐŝŐŶĂůƐ;ϰϮй ƐŝŐŶŝĮ ĐĂŶƚůǇ

33% somewhat), general grammar

(42% significantly, 40% somewhat)

and sentence structure (60% signifi-

cantly, 33% somewhat). Naturalness

of expression, text structure and

general vocabulary were also per-

ceived as having improved at least

somewhat by most students. The

“Students’ comments to

open-ended survey

questions highlighted some

reasons why audio record-

ings and email were so

popular, for example,

mentioning that feedback

was more detailed and

timely, and felt more indi-

vidualized than the feed-

back they were used to.”

21


Issue 63 21

ƌĞĂƐŽŶƐĨŽƌƚŚĞƐĞƉĞƌĐĞƉƟŽŶƐ ĂƐ

reported in open-ended survey

ƋƵĞƐƟŽŶƐ ĐĞŶƚƌĞĚĂƌŽƵŶĚƚŚĞĨĞĞĚͲ

back being more specific and placed

throughout the text, in contrast to

ƌĂƚŚĞƌŵŽƌĞƐƵŵŵĂƟǀ ĞĨĞĞĚďĂĐŬ

ĐŽŵŵĞŶƚƐƵƐƵĂůůǇǁ ƌŝƩĞŶĂƚƚŚĞ

ends of texts.

dŚĞĂĐĂĚĞŵŝĐǁ ƌŝƟŶŐƐŬŝůůƐŽĨĐƌŝƟĐĂů

thinking and self-ĞĚŝƟŶŐǁ ĞƌĞƐĞĞŶ

as having significantly improved by

over half of the students surveyed,

and as having somewhat improved

ďǇĂƌŽƵŶĚϰϬй >ĂƌŐĞŵĂũŽƌŝƟĞƐŽĨ

students also reported perceiving

improvement in logical argumenta-

ƟŽŶƐŬŝůůƐ;ϴϳ й ͿĂŶĚƚŚĞŝƌĂďŝůŝƚǇ

to engage in academic discourse

(80%). Overall, over three quar-

ters of the students surveyed

felt that LDF had been more

helpful in improving their aca-

demic skills related to essay

ǁ ƌŝƟŶŐƚŚĂŶƚŚĞŚĂŶĚ-ǁ ƌŝƩ ĞŶ

ĨĞĞĚďĂĐŬƚŚĞǇƚƌĂĚŝƟŽŶĂůůǇƌĞͲ

ceive.

Comparisons of students’ grades on

the essays for which they received

LDF and their grades on other essay

assignments supported the self-

ƌĞƉŽƌƚĞĚƉĞƌĐĞƉƟŽŶƐ ĂŶĚĐŽŶͲ

firmed that there was some real

improvement. On a grading system

out of 15 points, students’ essay

scores on the LDF assignment were

on average 2.22 points higher than

those received one year earlier and

on average 1.7 points higher than

those received on an assignment

ǁ ŝƚŚƚƌĂĚŝƟŽŶĂůĨĞĞĚďĂĐŬŝŶƚŚĞ

same term. The students’ essay

scores on an assignment with tradi-

ƟŽŶĂůĨĞĞĚďĂĐŬŝŶƐĂŵĞƚĞƌŵĂƐƚŚĞ

LDF assignment were on average

just 0.64 points higher than those

received one year earlier – smaller

improvements here would seem to

indicate that LDF had a role to play.

Discussion

These results show that learners

ƌĞƐƉŽŶĚĞĚƉŽƐŝƟǀ ĞůǇƚŽƚŚĞ>&ĂƉͲ

proach and that it was beneficial for

ƚŚĞŝƌǁ ƌŝƩ ĞŶǁ ŽƌŬThe findings thus

highlight LDF as a viable feedback

ƉƌĂĐƟĐĞŝŶƚŚŝƐĐŽŶƚĞǆƚ ǁ ŝƚŚĂĚͲ

vanced-level EAP students. Stu-

dents’ comments in response to

open-ĞŶĚĞĚƐƵƌǀ ĞǇƋƵĞƐƟŽŶƐŚŝŶƚ

at concepts which may underpin the

efficacy of the approach: LDF ad-

dresses issues of intelligibility,

‘authority’ over the texts, learner

ĂƵƚŽŶŽŵǇĂŶĚƉĞƌƐŽŶĂůŝƐĂƟŽŶ

These issues are not unique to EAP,

and thus the LDF approach may

ǁ ĂƌƌĂŶƚƉŝůŽƟŶŐŝŶŽƚŚĞƌĐŽŶƚĞǆƚƐ

too.

Students’ willing use of digital deliv-

ery modes for feedback can be prac-

ƟĐĂůĨŽƌƚĞĂĐŚĞƌƐ ǁ ŚŽĐĂŶŽŌĞŶ

provide more, and more detailed,

feedback more quickly than by

ǁ ƌŝƟŶŐĐŽŵŵĞŶƚƐďǇŚĂŶĚ /ƚĐĂŶ

also be efficient for tutors to give

feedback on selected aspects of the

ǁ ƌŝƟŶŐĚƵƌŝŶŐƚŚĞĚƌĂŌŝŶŐƐƚĂŐĞ

ƌĂƚŚĞƌƚŚĂŶĂƩĞŵƉƟŶŐƚŽĂĚĚƌĞƐƐ

all of the issues at once. Involving

students in the decisions about

the delivery of their feedback and

content may also help remove

some urgency from teachers to

agree on one ‘correct’ feedback

procedure, and can moreover

ĞŶŚĂŶĐĞĚŝī ĞƌĞŶƟĂƟŽŶďǇĂůůŽǁ Ͳ

ing for students to request feed-

back that suits their individual

strengths and weaknesses.

There are, of course, a few caveats

to these promising findings. For ex-

ample, students may not know

which aspects of their work to ask

ĂďŽƵƚ ŽƌŵĂǇŶŽƚďĞĂďůĞƚŽŝĚĞŶƟͲ

fy their own weaknesses, and lower-

level learners may not have suffi-

cient meta-language to pose effec-

Ɵǀ ĞĨĞĞĚďĂĐŬƋƵĞƐƟŽŶƐ /ĂĚĚƌĞƐƐ

some of these drawbacks below.

“Students’ willing use of digital

delivery modes for feedback

can be practical for teachers,

who can often provide more,

and more detailed, feedback

more quickly than by writing

comments by hand.”

22


Issue 63 22

dŝƉƐĨŽƌŝŵƉůĞŵĞŶƟŶŐ>&

LDF is easiest (I find) if students

submit their work electronically.

Decide which delivery modes you

wish to offer, and how students

ƐŚŽƵůĚƉŽƐĞƚŚĞƋƵĞƐƟŽŶƐ;Ğ Ő

footnotes, comments in margins,

or cover-sheet).

Allow any/only feedback delivery

modes that are workable in your

context. (i.e. perhaps not face-to-

ĨĂĐĞĐŽŶƐƵůƚĂƟŽŶŝĨǇŽƵŚĂǀ ĞϯϬϬ

students!)

Talk through the LDF process and

provide a handout to support stu-

ĚĞŶƚƐƚŚĞĮ ƌƐƚƟŵĞ;ƐͿƚŚĞǇǁ ŽƌŬ

with it.

ŽŵďŝŶĞ>&ǁ ŝƚŚǁ ƌŝƟŶŐǁ ŽƌŬͲ

shops, peer review, and other

ǁ ĂǇƐŽĨŐĞƫ ŶŐƐƚƵĚĞŶƚƐƚŚŝŶŬŝŶŐ

more carefully about their work,

to scaffold the LDF process.

Aim to provide feedback in a

ƟŵĞůǇŵĂŶŶĞƌĂŶĚĂůůŽǁ ĞŶŽƵŐŚ

ƟŵĞďĞƚǁ ĞĞŶĚƌĂŌƐƵďŵŝƐƐŝŽŶƐ

for students to thoroughly en-

gage with the feedback.

Give students guidance and/or

training on what makes for a

ŐŽŽĚƌĞƐƉŽŶƐĞƚŽƚŚĞǁ ƌŝƟŶŐƚĂƐŬ

at hand so that they can ask

about these points in their own

ǁ ƌŝƟŶŐ

If possible, provide students with

a grading matrix so they know

ǁ ŚĂƚĂƐƉĞĐƚƐŽĨƚŚĞŝƌǁ ƌŝƟŶŐƚŚĞǇ

ƐŚŽƵůĚƉĂǇĂƩĞŶƟŽŶƚŽ

Encourage students to pose both

ƐƉĞĐŝĮ ĐƋƵĞƐƟŽŶƐ;Ğ Őt ŚŝĐŚ

word is most appropriate here?)

ĂŶĚŐĞŶĞƌĂůƋƵĞƐƟŽŶƐ;Ğ Ő/ƐƚŚĞ

vocabulary I’m using formal

enough?).

If necessary and feasible, allow

ĨĞĞĚďĂĐŬƋƵĞƐƟŽŶƐŝŶƐƚƵĚĞŶƚƐ

L1.

For audio recordings or emailed

feedback, insert line numbers to

be able to refer to

specific parts of the

text.

For audio recordings,

you can email stu-

dents an .mp3 file, or

use free recording

websites such as

www.vocaroo.com

and send students the

links to their feed-

back.

Remind students to

ŚĂǀ ĞƚŚĞŝƌǁ ƌŝƩ ĞŶ

text at hand whilst

listening to / reading

the feedback.

Note down common

problems that stu-

dents do not request feedback

on, and use these to plan future

teaching or workshops.

Sit back and enjoy a cup of tea, as

>&ǁ ŝůůƉƌŽďĂďůǇƐĂǀ ĞǇŽƵƟŵĞ

(You might even want to throw

out your red pens!)

Foto

:Sh

aro

nH

artl

e

www.vocaroo.com

23


Issue 63 23

Date Presenter Title

19.12.17

Felicity O’Dell andRussell Whitehead

' Ğƫ ŶŐŝƚƌŝŐŚƚ ŐĞƫ ŶŐŝƚǁ ƌŽŶŐĂƚŽƵƌŽĨƚŚĞ>dϭϮϯƚĞƐƟŶŐƉƌŝŶĐŝƉůĞƐ

This webinar with Felicity O’Dell and Russell Whitehead willĚŝƐĐƵƐƐ>dϭϮϯ ƐůŝƐƚŽĨƚĞŶƉƌŝŶĐŝƉůĞƐĨŽƌŐŽŽĚ ƚĞƐƟŶŐƐŚĂƌŝŶŐplenty of examples of successful and unsuccessful items andtasks. This promises to be a useful round-up for newcomersand experienced testers alike.

Gudrun EricksondŚĞƵĂů&ƵŶĐƟŽŶŽĨƐƐĞƐƐŵĞŶƚ–Enhancing Learning and Equity

dŚŝƐǁ ĞďŝŶĂƌǁ ŝůůĨŽĐƵƐŽŶƚŚĞĚƵĂůĨƵŶĐƟŽŶŽĨĂƐƐĞƐƐŵĞŶƚ whether in classrooms or large scale contexts, namely toenhance learning as well as fairness and equity. Gudrun willdiscuss the balancing act between the two, emphasizingĐŽŵŵŽŶƉƌŝŶĐŝƉůĞƐƚŚĂƚŶĞĞĚƚŽŐƵŝĚĞƉůĂŶŶŝŶŐƉƌĂĐƟĐĞƐĂŶĚmaterials at individual as well as structural levels.

26.02.18

The webinar can be accessed on the day at: ŚƩ Ɖ ŝĂƚĞŇĂĚŽďĞĐŽŶŶĞĐƚ ĐŽŵ ƚĞĂƐŝŐǁ ĞďŝŶĂƌƐand is open to everybody. It will be recorded, and the recording will be accessible to everybody for aǁ ĞĞŬŽŶƚŚĞd^/' ǁ ĞďƐŝƚĞĂŌĞƌǁ ŚŝĐŚŝƚǁ ŝůůŽŶůǇďĞĂǀ ĂŝůĂďůĞƚŽ/d&>ŵĞŵďĞƌƐ d^/' ǁ ĞďŝŶĂƌƐǁ ŝůůďĞĐŽŶƟŶƵŝŶŐŝŶϮϬϭϳ ĂŶĚĚĞƚĂŝůƐǁ ŝůůďĞĂŶŶŽƵŶĐĞĚŽŶƚŚĞd^/' ǁ ĞďƐŝƚĞ

TEASIGwebinars

2017

http://iatefl.adobeconnect.com/teasigwebinars

24


Issue 63 24

ŌĞƌƐĞǀ ĞƌĂůǇĞĂƌƐŝŶƚĞĂĐŚĞƌĞĚƵͲĐĂƟŽŶ/ĐŽŶƟŶƵĞƚŽďĞĂŵĂǌĞĚďǇthe trainee teachers in my seminargroups. They consistently bring withthem a huge amount of knowledge,ǁ ŚĞƚŚĞƌůŝŶŐƵŝƐƟĐůŝƚĞƌĂƌǇŽƌĐƵůƚƵƌͲal. They also bring a wealth of skillsin areas ranging from tutoring toĂĚŵŝŶŝƐƚƌĂƟŽŶ &ŝŶĂůůǇĂŶĚŝŶŵǇview crucially, for the purposes oftraining new teachers who are alertand aware, they have of courseclocked up thousands of hours aslearners in school contexts them-selves. These vital experiences(good lessons and bad, inspiringtopics and less inspiring ones) canvery usefully be put to work in arange of ways in our seminars –from examininglearner (andƚĞĂĐŚĞƌ ͿŵŽƟǀ ĂͲƟŽŶ ĨŽƌĞǆĂŵƉůĞƚŽƌĞŇĞĐƟŶŐŽŶclassroom meth-ods which learnersappreciate. Thisprovides an idealƐŝƚƵĂƟŽŶĨŽƌƉƌŽͲĚƵĐƟǀ ĞƐĞŵŝŶĂƌwork which focuses on classroomƉƌĂĐƟĐĞ;ƐͿ

dŚŝƐĨŽĐƵƐŽŶĐůĂƐƐƌŽŽŵƉƌĂĐƟĐĞŶĞĐĞƐƐŝƚĂƚĞƐĂƉƌĂŐŵĂƟĐĂƉƉƌŽĂĐŚto learning how to teach, and weƚŚĞƌĞĨŽƌĞǁ ŽƌŬǀ ĞƌǇƉƌĂĐƟĐĂůůǇŝŶour seminar sessions. It goes with-out saying that, as a universityƚĞĂĐŚĞƌĞĚƵĐĂƟŽŶĚĞƉĂƌƚŵĞŶƚ ǁ Ğconsistently draw on research find-ŝŶŐƐĂŶĚĂĐĐĞƉƚĞĚďĞƐƚƉƌĂĐƟĐĞŝŶour work, but we nevertheless al-ǁ ĂǇƐŚĂǀ ĞŝŶŵŝŶĚƚŚĞƵůƟŵĂƚĞtransfer to the classroom and possi-ďůĞŝŵƉůĞŵĞŶƚĂƟŽŶǁ ŝƚŚƐĞĐŽŶĚĂƌǇ

school learners. To this end, train-ee teachers are regularly asked todesign a task or sequence of tasksfor one or more lessons in whichƚŚĞǇƉƵƚƚŚĞŝƌƚŚĞŽƌĞƟĐĂůƌĞĂĚŝŶŐƐŝŶƚŽƉƌĂĐƟĐĞ

Against this backdrop we engagewith many key issues of upper sec-ondary (language) learning, lookinginto principles of lesson-planningand curricula, the four skills, andtask- and competence-orientedlearning. Within this demandingmenu of important angles, one ma-jor area it's easy to feel over-whelmed by as a beginning teacheris surely that of assessment andƚĞƐƟŶŐdĞƐƟŶŐŐĞƚƐĂůŽƚŽĨďĂĚ

press: pupilshave on oc-casion beenknown tocomplainabout thevolume ofmaterial tobe learned,and teachersabout all the

marking theyhave to do. For this reason, as wellas many others, I feel it's importantƚŽƉƵƚĂƉŽƐŝƟǀ ĞƐƉŝŶŽŶĂƐƐĞƐƐŵĞŶƚand to show trainee teachers thatassessment can be a manageable,ƌĞǁ ĂƌĚŝŶŐĂŶĚƵůƟŵĂƚĞůǇŝŶƚĞƌͲĞƐƟŶŐĂƐƉĞĐƚŽĨƚĞĂĐŚŝŶŐĂŶĚůĞĂƌŶͲing.

We typically start by looking at clas-ƐŝĐƚĞƐƚĚĞƐŝŐŶĂŶĚĨŽƌŵĂƚƐ ŝĚĞŶƟĨǇͲing the markers of a successful test,ĂŶĚĂƚƚŚĞƐĂŵĞƟŵĞĐŽŶƚĞǆƚƵĂůŝƐͲŝŶŐŽƵƌŝŶƚƵŝƟǀ ĞƌĞĂĐƟŽŶƐĂŐĂŝŶƐƚĂďĂĐŬŐƌŽƵŶĚŽĨƚŚĞƚŚĞŽƌĞƟĐĂůƵŶͲ

Lynn Williams

grew up and went to school nearManchester, England. She studiedfor a language degree at universi-

ty and later added a teachingƋƵĂůŝĮ ĐĂƟŽŶ ŚĞŶŽǁ ƚĞĂĐŚĞƐ

English language and literature ata Swiss grammar school and alsoworks as a teacher trainer at theW, &, E t ŝŶĂƐĞů ŚĞŝƐƉĂƌƟĐƵͲ

larly interested in assessmentƉƌĂĐƟĐĞƐ ƚŚĞƚĞĂĐŚŝŶŐŽĨůŝƚĞƌĂͲƚƵƌĞĂŶĚŽƉƉŽƌƚƵŶŝƟĞƐĨŽƌŝŶĚŝͲ

ǀ ŝĚƵĂůŝƐĂƟŽŶůǇŶŶ ǁ ŝůůŝĂŵƐΛ Ĭ Ŷǁ ĐŚ

Giving testing some good press for a changeAssessment as learning check - a meaningful paradigm shift

for trainee teachers

“The link between

teaching and assessment

is not always clear at first

sight and sometimes

needs to be made more

explicit.”

25


Issue 63 25

ĚĞƌƉŝŶŶŝŶŐƐŽĨƚĞƐƚĚĞƐŝŐŶ ŌĞƌĐŽŶƐŝĚĞƌŝŶŐƚŚĞŽƌĞƟĐĂůĂƐƉĞĐƚƐƐƵĐŚas validity and reliability, we moveon to look at seemingly banal butƵůƟŵĂƚĞůǇĞƐƐĞŶƟĂůƉƌĂĐƟĐĂůĂƐƉĞĐƚƐlike concise and precise rubric for-ŵƵůĂƟŽŶŽƌĞǀ ĞŶŶƵŵďĞƌŝŶŐůŝŶĞƐŝŶa reading text for ease of orienta-ƟŽŶǁ ŚĞŶĚŝƐĐƵƐƐŝŶŐĂƚĞƐƚŝŶĐůĂƐƐat a later date.

Furthermore, I encourage traineeteachers to engage with principlesof backwash, my message beingĞƐƐĞŶƟĂůůǇ

KŶĐĞǇŽƵĂƌĞƐĂƟƐĮ ĞĚǁ ŝƚŚƚŚĞtest, use it to inform students /shape your teaching / studentƉƌĞƉĂƌĂƟŽŶǁ ŝƚŚŽƵƚĚŝƐĐůŽƐŝŶŐŬĞǇaspects.

^ŽŵĞƚŚŝŶŐŝŶƚĞƌĞƐƟŶŐĂŶĚƉĞƌŚĂƉƐ

ƵŶƐƵƌƉƌŝƐŝŶŐƚŚĂƚ/Ζǀ ĞŶŽƟĐĞĚŽǀ Ğƌthe years is that trainee teachersfrequently associate assessmentwith tests. In response to this obser-ǀ ĂƟŽŶ /ŚĂǀ ĞƚǇƉŝĐĂůůǇƐƚĂƌƚĞĚĂseminar session on assessment andƚĞƐƟŶŐǁ ŝƚŚĂĨƵŶĚĂŵĞŶƚĂůĚŝƐĐƵƐͲsion on the nature of assessmentwith a task like this:

A test is an assessmentbut an assessment is notnecessarily a test!

Discuss what you under-stand by this and be pre-pared to feedback in theseminar group.

As we know, the range ofŽƉƟŽŶƐǁ ŚŝĐŚŐŽďĞǇŽŶĚthe classic 'test' scenariois huge, ranging from quizƋƵĞƐƟŽŶƐĂƚƚŚĞĞŶĚŽĨĂlesson to self-Ğǀ ĂůƵĂƟŽŶ

according to prescribed learningŽďũĞĐƟǀ ĞƐ dƌĂŝŶĞĞƚĞĂĐŚĞƌƐĂƌĞtherefore encouraged to trawlthrough their (language) learningďŝŽŐƌĂƉŚŝĞƐƚŽŝĚĞŶƟĨǇƉŽƐƐŝďůĞĂƐͲƐĞƐƐŵĞŶƚƐĐĞŶĂƌŝŽƐ ƉĂƌƟĐƵůĂƌůǇthose that explode the classic canonŽĨĐŽŶƚƌŽůůĞĚŐƌĂŵŵĂƌƉƌĂĐƟĐĞĞǆͲercises. In a further step, the train-ees devise an appropriate assess-ment strategy for a teaching unit

they are developing andƌĞŇĞĐƚŽŶƚŚĞƌĂƟŽŶĂůĞbehind their choice.

However, in my experi-ence, the link betweenteaching and assessmentis not always clear at first ƐŝŐŚƚĂŶĚƐŽŵĞƟŵĞƐneeds to be made more

explicit. For example, trainee teach-ĞƌƐĨƌĞƋƵĞŶƚůǇ;ĂŶĚũƵƐƟĮ ĂďůǇͿƚĂŬĞĂƚŽƉŝĐŽƌƉĂƌƟĐƵůĂƌĂƌƟĐůĞŽƌĮ ůŵĐůŝƉĂƐƚŚĞƐƚĂƌƟŶŐƉŽŝŶƚĨŽƌĂƉŽƚĞŶͲƟĂůƚĞĂĐŚŝŶŐƵŶŝƚǁ ŝƚŚŽƵƚŶĞĐĞƐƐĂƌŝͲly having any special learning objec-Ɵǀ ĞŝŶŵŝŶĚ– they simply trust theirŝŶƐƟŶĐƚƐ ŬŶŽǁ ŝŶŐƚŚĂƚŝƚŝƐůŝŬĞůǇƚŽŵŽƟǀ ĂƚĞƵƉƉĞƌƐĞĐŽŶĚĂƌǇůĞĂƌŶĞƌƐor that it serves to exemplify a par-ƟĐƵůĂƌůĂŶŐƵĂŐĞƉŽŝŶƚǁ ŝƚŚŝŶƚŚĞĐŽŶƚĞǆƚŽĨĂŶĂƵƚŚĞŶƟĐƚĞǆƚ– bin-go! And good for them!

In my seminars, therefore, as well asŝŶŝŶĚŝǀ ŝĚƵĂůĐŽĂĐŚŝŶŐƐĞƫ ŶŐƐĂŶĚŝŶǁ ƌŝƩ ĞŶĨĞĞĚďĂĐŬŽŶĚƌĂŌƵŶŝƚƐ /seek to guide trainee teachers to-wards examining the underlyingƌĂƟŽŶĂůĞĨŽƌƚŚĞƵŶŝƚ ƚŚĞƌĞĂƐŽŶƐǁ ŚǇƚŚĞǇŚĂǀ ĞĐŚŽƐĞŶƚŚĞƉĂƌƟĐƵůĂƌtopic, text or task. Having estab-lished the competences orknowledge they want their uppersecondary learners to acquire, itbecomes easier for them to thinkĂďŽƵƚƚŚĞŝƌ;ŝŵƉůŝĐŝƚͿŽďũĞĐƟǀ ĞƐĨŽƌthe unit and, consequently, alsoabout how they can then check thatƚŚĞƐĞŽďũĞĐƟǀ ĞƐŚĂǀ ĞďĞĞŶachieved. This is the point at whichǁ ĞƚƵƌŶŽƵƌĂƩĞŶƟŽŶƚŽĂƐƐĞƐƐŵĞŶƚprocedures. Trainees realise –ƐŽŵĞƟŵĞƐǁ ŝƚŚĂũŽůƚ ƐŽŵĞƟŵĞƐwith a degree of disappointment,ďƵƚŵŽƌĞŽŌĞŶƚŚĂŶŶŽƚǁ ŝƚŚƌĞůŝĞĨ– that a classic 'test' might not benecessary to check their learners'progress. Or it might not be feasible,plausible or manageable within cer-tain constraints. Or, quite simply, itmight not be the best way of check-ing learning.

"So what is the best way?" my train-

26


Issue 63 26

ees then start to wonder, and I'malways nothing short of thrilled atthis point. It means we are breakingaway from the mindset that seesteachers replicate the kind of testthey themselves sat at school orthose that they might find through external searches for material,ŽŌĞŶĚĞƐŝŐŶĞĚƚŽĮ ƚĂůůƚƚŚŝƐpoint, let it be said that a good testis a good test, no quibbles there(and I've taken, set and correctedenough of them by now!), but weĂƌĞŶŽǁ ĂƚĂũƵŶĐƚƵƌĞƐĞƫ ŶŐŽƵƚinto new territory – namely the re-ĂůŝƐĂƟŽŶƚŚĂƚĂŐŽŽĚĂƐƐĞƐƐŵĞŶƚdoesn't need to be a test at all…

In my opinion, the 'best' way to as-sess a unit of work is at the sameƟŵĞĂŶŽďũĞĐƟǀ ĞĂŶĚƐƵďũĞĐƟǀ ĞĚĞĐŝƐŝŽŶ KďũĞĐƟǀ ĞďĞĐĂƵƐĞŵƵĐŚŽĨƚŚĞůŝƚĞƌĂƚƵƌĞŽŶƚĞƐƟŶŐƌŝŐŚƚůǇargues that an assessment shouldmirror the learning as far as thismakes sense (i.e. it would be odd tofollow up a series of discussion les-sons to develop fluency with a dis-crete-item grammar test, and byƚŚĞƐĂŵĞƚŽŬĞŶ ǁ ƌŝƟŶŐĂĨŽƌŵĂůůĞƩ ĞƌŵĂǇŶŽƚŶĞĐĞƐͲsarily be the best way tocheck learners' understand-ing of a range of new gram-mar points). Once traineeƚĞĂĐŚĞƌƐŚĂǀ ĞŝĚĞŶƟĮ ĞĚwhat they consider to bethe best way to check learn-ing of the unit they have puttogether – i.e. that whichbest aligns with the learningthey are striving for and theoverall competence-oriented outcome they areaiming at – they are encour-

aged to engage with a variety ofƋƵĞƐƟŽŶƐƚŽŚĞůƉƚŚĞŵƌĞĮ ŶĞƚŚĞŝƌassessment strategy. These mightƉƌŽĚƵĐƟǀ ĞůǇŝŶĐůƵĚĞƚŚĞĨŽůůŽǁ ŝŶŐ

Does the assessment methodreflect the style of work under-taken? (Does it assess thelearning which has gone be-fore? Are learners well-prepared to respond to thetask? In what way will itdemonstrate learners' grasp ofƚŚĞƚŽƉŝĐŝŶƋƵĞƐƟŽŶ Ϳ

Is the assessment task manage-able? (Timing, level, resources,teacher feedback)

/ƐƚŚĞĂƐƐĞƐƐŵĞŶƚƚĂƐŬŵŽƟͲǀ ĂƟŶŐ ;ŽĞƐŝƚŽī ĞƌůĞĂƌŶĞƌƐchoices? Does it appeal todifferent interests? Does it take up a relevant topic?)

KŶĐĞƚƌĂŝŶĞĞƚĞĂĐŚĞƌƐŚĂǀ ĞΖǀ ĞƩ ĞĚΖtheir assessment procedure tocheck its viability in these areas,ƚŚĞǇĂƌĞŝŶĂƉŽƐŝƟŽŶƚŽŵĂŬĞĮ ŶĂůadjustments and refine their assess-ment task. In a final step, we typi-

cally share teaching units and therelated assessments at the end ofthe semester. There is no small de-gree of pride during this phase and,having had the privilege of guidingƚƌĂŝŶĞĞƚĞĂĐŚĞƌƐŝŶƚŚĞƌĞĂůŝƐĂƟŽŶof their proposed assessment tasks,I understand their pride! I also hopethat a few key tenets will stay withƚŚĞŵĂŌĞƌŽƵƌũŽŝŶƚǁ ŽƌŬŽŶĂƐƐĞƐƐͲment strategies:

^ĞĞƚŚĂƚƚĞƐƟŶŐŝƐĂďŽƵƚŵŽƌĞƚŚĂŶũƵƐƚĂƩ ĂŝŶŝŶŐƐĐŽƌĞƐ Make tests worthwhile to allŝŶǀ Žůǀ ĞĚ ŵŽƟǀ ĂƟŶŐĨŽĐƵƐĞĚ ƌĞĂůŝƐƟĐŝŶĨŽƌŵĂƟǀ Ğ

>ŝŶŬƚĞƐƟŶŐƚŽĐůĂƐƐƌŽŽŵǁ ŽƌŬas far as possible scores. Maketests worthwhile to all class-ƌŽŽŵǁ ŽƌŬĂŶĚƚĞƐƟŶŐ

These are principles which guide myown classroom work, leading tolearning which I hope is all the rich-er, sustainable and more enjoyablefor it.

27


Issue 63 27

Assessment in the upper secondary ELT classroom

(introductory handout for trainee teachers)

&ŽƌŵƐŽĨƚĞƐƟŶŐĂŶĚĂƐƐĞƐƐŵĞŶƚ

It is important to note that an assessment does not necessarily need to be a test. It is equally possibleƚŽĐŚĞĐŬǁ ŚĞƚŚĞƌƐƚƵĚĞŶƚƐŚĂǀ ĞĂĐŚŝĞǀ ĞĚƚŚĞƌĞůĞǀ ĂŶƚůĞĂƌŶŝŶŐŽďũĞĐƟǀ ĞƐĨŽƌĂƵŶŝƚŝŶĂĚŝī ĞƌĞŶƚĂƐͲsessment set-ƵƉƚŚĂŶĂĨŽƌŵĂůǁ ƌŝƩ ĞŶƚĞƐƚ;ŽƌƚŚĞĞƋƵŝǀ ĂůĞŶƚƐƉĞĂŬŝŶŐĂƐƐĞƐƐŵĞŶƚͿ ŽŵĞŝĚĞĂƐŝŶͲclude:

- ĐŽŶĐĞƉƚƋƵĞƐƟŽŶƐ;ǁ ƌŝƩ ĞŶŽƌŽƌĂůͿŝŶĚŝǀ ŝĚƵĂůůǇŝŶŐƌŽƵƉƐŽƌŝŶƚŚĞƉůĞŶĂƌǇ;Ăůǁ ĂǇƐǁ ŝƚŚĨĞĞĚďĂĐŬĂŶĚƚĞĂĐŚĞƌĐůĂƌŝĮ ĐĂƟŽŶ ĞǆƉĂŶƐŝŽŶǁ ŚĞƌĞŶĞĐĞƐƐĂƌǇĂŶĚĂƉƉƌŽƉƌŝĂƚĞͿ

- self-Ğǀ ĂůƵĂƟŽŶ;ďĂƐĞĚŽŶƚĞĂĐŚĞƌƉƌŽŵƉƚƐͿ

- ƉĞĞƌĨĞĞĚďĂĐŬ;ĨŽůůŽǁ ĞĚďǇĚŝƐĐƵƐƐŝŽŶŝŶƉůĞŶĂƌǇĐŽŶƐƵůƚĂƟŽŶǁ ŝƚŚƚŚĞƚĞĂĐŚĞƌͿ

- ǁ ƌŝƩ ĞŶŽƌŽƌĂůĨĞĞĚďĂĐŬĐŽŵŵĞŶƚƐĨƌŽŵƚŚĞƚĞĂĐŚĞƌ

- ĂŶĂƉƉůŝĐĂƟŽŶŽĨƚŚĞĐŽŵƉĞƚĞŶĐĞƐĂĐƋƵŝƌĞĚŝŶĂŶĞǁ ƐĞƫ ŶŐ;ƚƌĂŶƐĨĞƌͿ

- a product (poster, mind map, reader's guide, handout, flyer, theatre programme…)

Principles of assessment

ůůĨŽƌŵƐŽĨĂƐƐĞƐƐŵĞŶƚŵƵƐƚĚĞŵŽŶƐƚƌĂƚĞŐĞŶƵŝŶĞǀ ĂůŝĚŝƚǇĂŶĚƐƚĂŶĚƵƉƚŽƐĐƌƵƟŶǇdŚĞƌĞĨŽƌĞ

- ĂĚŚĞƌĞƚŽƚŚĞƐƚĂŶĚĂƌĚƉƌŝŶĐŝƉůĞƐŽĨƚĞƐƟŶŐ;ƌĞůŝĂďŝůŝƚǇǀ ĂůŝĚŝƚǇĨĂŝƌŶĞƐƐͿĂŶĚŽī ĞƌŽƉƉŽƌƚƵŶŝƟĞƐĨŽƌƉŽƐŝƟǀ ĞďĂĐŬǁ ĂƐŚ

- communicate test requirements, format and criteria early on

- ďĞƉƌĞƉĂƌĞĚƚŽĂŶƐǁ ĞƌĂŶǇƋƵĞƐƟŽŶƐ;ĂŶĚƌĞ-think as and when necessary)

WƌĂĐƟĐĂďŝůŝƚǇ

dĞƐƚƐƐŚŽƵůĚďĞƐƚƌĂŝŐŚƞŽƌǁ ĂƌĚĨŽƌƚĞĂĐŚĞƌƐƚŽĂĚŵŝŶŝƐƚĞƌĂŶĚƐƚƵĚĞŶƚƐƚŽƚĂŬĞĂŶĚƐŚŽƵůĚŶŽƚƉƌĞͲsent unnecessary challenges in marking.

t ƌŝƩ ĞŶƚĞƐƚƐƐŚŽƵůĚ

- test what has been learned

- ƌĞŇĞĐƚƉƌĞǀ ŝŽƵƐƚĞĂĐŚĞƌŝŶĨŽƌŵĂƟŽŶŽŶƚŚĞƚĞƐƚ

- Žī ĞƌĐůĞĂƌŝŶƐƚƌƵĐƟŽŶƐĂŶĚŝŶĚŝĐĂƚĞŵĂƌŬƐƉĞƌƋƵĞƐƟŽŶ

- ideally, present a variety of tasks

- give all students a fair chance, and strong students the chance to excel

&ĞĞĚďĂĐŬ ĐŽƌƌĞĐƟŽŶƐĂƌĞĞƐƐĞŶƟĂůĂŶĚƐŚŽƵůĚ

- clearly show students where they went wrong, help address these points, and offer students paths for development

- report on performance in content terms, insofar as this is being tested

Suggested further reading ůĚĞƌƐŽŶ : ůĂƉŚĂŵ Θt Ăůů ;ϭϵϵϱͿ >ĂŶŐƵĂŐĞdĞƐƚŽŶƐƚƌƵĐƟŽŶĂŶĚǀ ĂůƵĂƟŽŶ ĂŵďƌŝĚŐĞhW

ŽƵŐůĂƐ ;ϮϬϭϬͿ hŶĚĞƌƐƚĂŶĚŝŶŐůĂŶŐƵĂŐĞƚĞƐƟŶŐ>ŽŶĚŽŶ , ŽĚĚĞƌĚƵĐĂƟŽŶ

&ƵůĐŚĞƌ' ;ϮϬϭϬͿ WƌĂĐƟĐĂůůĂŶŐƵĂŐĞƚĞƐƟŶŐ>ŽŶĚŽŶ , ŽĚĚĞƌĚƵĐĂƟŽŶ

, ƵŐŚĞƐ ;ϮϬϬϯ Ϳ dĞƐƟŶŐĨŽƌ>ĂŶŐƵĂŐĞdĞƐƟŶŐ;ϮŶĚĞĚ ͿĂŵďƌŝĚŐĞĂŵďƌŝĚŐĞhŶŝǀ ĞƌƐŝƚǇWƌĞƐƐ

D ĐEĂŵĂƌĂ d ;ϮϬϬϬͿ >ĂŶŐƵĂŐĞdĞƐƟŶŐKǆĨŽƌĚ KǆĨŽƌĚhŶŝǀ ĞƌƐŝƚǇWƌĞƐƐ

/ƚŝƐĂůƐŽǁ ŽƌƚŚĐŽŶƐƵůƟŶŐƚŚĞŐƵŝĚĞůŝŶĞƐĨŽƌŐŽŽĚƉƌĂĐƟĐĞŽŶůĂŶŐƵĂŐĞƚĞƐƟŶŐĂŶĚĂƐƐĞƐƐŵĞŶƚŝƐƐƵĞĚďǇƚŚĞƵƌŽƉĞĂŶƐƐŽĐŝa-ƟŽŶĨŽƌ>ĂŶŐƵĂŐĞdĞƐƟŶŐĂŶĚƐƐĞƐƐŵĞŶƚ;>dͿ ƐĞĞwww.ealta.ue.org .

www.ealta.ue.org

28


Issue 63 28

/ŶƚƌŽĚƵĐƟŽŶ

/ŶŚŝƐϮϬϬϰĂƌƟĐůĞŝŶLanguage As-sessment Quarterly ĂĚǀ ŽĐĂƟŶŐƚŚĞneed to “broaden, deepen and con-solidate” many of our ideas aboutůĂŶŐƵĂŐĞƚĞƐƟŶŐƵŵŵŝŶŐŵĂŬĞƐthe convincing argument that moreresearch is needed on the role ofƐƚĂŬĞŚŽůĚĞƌƐŝŶůĂŶŐƵĂŐĞƚĞƐƟŶŐcontexts and areas that have tradi-ƟŽŶĂůůǇďĞĞŶŽǀ ĞƌůŽŽŬĞĚ;ƵŵŵŝŶŐ2004, p.3). It can be successfullyargued that one of these neglectedareas is Mexico. Mexico seems tofind itself in the paradoxical situa-ƟŽŶŽĨŵĂŶǇ>ĂƟŶŵĞƌŝĐĂŶĐŽƵŶͲtries which, while witnessing a pro-nounced increase in demand forhigh quality English language in-ƐƚƌƵĐƟŽŶĂŶĚĂƐƐĞƐƐŵĞŶƚ ŚĂǀ Ğfailed to produce a significant body ŽĨƌĞƐĞĂƌĐŚŝŶǀ ĞƐƟŐĂƟŶŐƚŚĞƐƉĞĐŝĮ Đvariables that help to define the uniqueness of their contexts.

Without doubt, one of these varia-bles is the students or candidateswho actually take language tests.Their story is largely untold, and inƌĞĐĞŶƚůĂŶŐƵĂŐĞƚĞƐƟŶŐůŝƚĞƌĂƚƵƌĞŝƚis difficult to find an issue that more scholars seem to agree on than theidea that candidates are among themost important – yet neglected –ƐƚĂŬĞŚŽůĚĞƌƐŝŶůĂŶŐƵĂŐĞƚĞƐƟŶŐShohamy (2001) perceives that “it isthrough the voices of test takerswho report on their experiences andconsequences that the features ofƚŚĞƵƐĞŽĨƚĞƐƚƐĐĂŶďĞŝĚĞŶƟĮ ĞĚ zĞƚ ŝŶƚŚĞƚĞƐƟŶŐůŝƚĞƌĂƚƵƌĞƚĞƐƚƚĂŬĞƌƐĂƌĞŽŌĞŶŬĞƉƚƐŝůĞŶƚ ƚŚĞŝƌpersonal experiences are not heardor shared” (p.7). Cumming (2004)

maintains that “serious considera-ƟŽŶŽĨƚŚĞƵƐĞƐŽĨůĂŶŐƵĂŐĞĂƐƐĞƐƐͲŵĞŶƚƌĞƋƵŝƌĞƐĂĚŽƉƟŶŐƌĞƐĞĂƌĐŚŵĞƚŚŽĚƐƚŚĂƚŝŶǀ ĞƐƟŐĂƚĞƉĞŽƉůĞƐĂƫ ƚƵĚĞƐ ďĞůŝĞĨƐ ĐƵůƚƵƌĂůǀ ĂůƵĞƐ ĂŶĚǁ ĂǇƐŽĨŝŶƚĞƌĂĐƟŶŐ; Ϳ ƵĐŚinquiry is indispensable for under-standing why people perform theways they do in language assess-ment, and thus necessary for valida-ƟŽŶ ;Ɖ ϵͿ

The purpose of this study, there-fore, was to give free rein to theneglected voices of test candidatesŝŶŽŶĞƉĂƌƟĐƵůĂƌĐŽŶƚĞǆƚ ƐMcNamara and Roever (2006) insist, ůĂŶŐƵĂŐĞƚĞƐƟŶŐŚĂƐĂƌĞĂůŝŵƉĂĐƚon real people's lives” (p.8). Thisimpact starts with the stakeholderswho are immediately affected by the test, i.e. test candidates and testdevelopers, and extends outward tosociety at large. This impact alsoimplies a significant amount of re-sponsibility on the part of test de-velopers to ensure that the teststhey write and administer are asvalid and reliable as possible.

One of the most valuable tech-niques for helping test developersto measure test validity is by listen-ing to candidates’ voices. CandidateƉĞƌĐĞƉƟŽŶƐ ĨĞĞůŝŶŐƐ ƉŽŝŶƚƐŽĨǀ ŝĞǁ Ăƫ ƚƵĚĞƐ ŽƉŝŶŝŽŶƐĂŶĚƐƵŐͲŐĞƐƟŽŶƐ ƚĂŬĞŶƚŽŐĞƚŚĞƌĐĂŶƐĞƌǀ ĞĂƐĞǀ ŝĚĞŶĐĞŽĨƚŚĞƉŽƐŝƟǀ ĞĂŶĚŶĞŐͲaƟve consequences of tests. In ad-ĚŝƟŽŶ ĨĞĞĚďĂĐŬĨƌŽŵĐĂŶĚŝĚĂƚĞƐcan serve as the impetus for discus-sions that can, and should, be hap-pening among a variety of stake-holders (Madaus, in press, as citedin Shohamy, 2001, p.149). Enlarging

David Ewing Ryan

is a full-ƟŵĞƉƌŽĨĞƐƐŽƌĂƚƚŚĞUniversidad Veracruzana (UV),Mexico, where he conductsƌĞƐĞĂƌĐŚŽŶůĂŶŐƵĂŐĞƚĞƐƟŶŐand serves as Chief Editor ofys Z ƚŚĞhs ƐƟĞƌĞĚ-suiteŽĨŶŐůŝƐŚůĂŶŐƵĂŐĞĐĞƌƟĮ ĐĂƟŽŶtests.

[email protected].

“En mi humilde opinión”1 …listening to Mexican students’perceptions of an English language proficiency test

29


Issue 63 29

the dialogue in this way can helpfurther promote not just the validityof individual tests, but also of thetest system ǁ ŚŝĐŚŶĞĞĚƐƚŽĐŽŶƟŶͲually “encourage testers, teachers,test takers, and the public at largeƚŽƋƵĞƐƟŽŶƚŚĞƵƐĞƐŽĨƚĞƐƚƐ ƚŚĞmaterials they are based on, and toĐƌŝƟƋƵĞƚŚĞǀ ĂůƵĞƐĂŶĚďĞůŝĞĨƐŝŶŚĞƌͲent in them” (Shohamy, p.131).

dŚĞĂƌƟĐůĞŚĂƐƐŝǆƉĂƌƚƐ WĂƌƚϭŽƵƚͲlines the goal of the study. Part 2summarises the EXAVER English lan-ŐƵĂŐĞĐĞƌƟĮ ĐĂƟŽŶƚĞƐƚƐ ǁ ŚŝĐŚƐĞƌǀ ĞĚĂƐƚŚĞƉƌĂĐƟĐĂůĐŽŶƚĞǆƚŽĨthe study. Part 3 explains the meth-odology of the study. Parts 4 and 5Žī ĞƌƌĞƐƉĞĐƟǀ ĞůǇĂŶŽǀ Ğƌǀ ŝĞǁ ĂŶĚa discussion of the findings. Finally, Part 6 offers some general conclu-sions.

1. Goal of the study

The study fo-cused specifi-cally on whatBachman andPalmer (1996)consider asone of thethree waysthat languagetests have adirect impact on test candidates,namely, the consequences that can-didates experience as a result ofpreparing for and taking these tests(p.31). In order to measure this im-pact, it was necessary to liberatethe voices of the candidates whoƉĂƌƟĐŝƉĂƚĞĚŝŶƚŚĞƐƚƵĚǇĂŶĚƚŚŝƐbecame the study’s primary goal.This was accomplished, first, by so-

ůŝĐŝƟŶŐĐĂŶĚŝĚĂƚĞŽƉŝŶŝŽŶƐĂďŽƵƚƚŚĞprocesses of preparing for and tak-ŝŶŐƚŚĞƚĞƐƚ ĂŶĚƚŚĞŶďǇƐŽůŝĐŝƟŶŐƚŚĞŝƌƐƵŐŐĞƐƟŽŶƐŽŶŚŽǁ ƚŚĞƐĞƉƌŽͲcesses might be improved.

Ϯ WƌĂĐƟĐĂůĐŽŶƚĞǆƚŽĨƐƚƵĚǇƚŚĞEXAVER English LanguageĞƌƟĮ ĐĂƟŽŶdĞƐƚƐ

Ϯ ϭ' ĞŶĞƌĂůĚĞƐĐƌŝƉƟŽŶ

EXAVER is the name ofthe tests used as the ba-sis of the study, and re-ĨĞƌƐƚŽĂƟĞƌĞĚ-suite ofŶŐůŝƐŚůĂŶŐƵĂŐĞĐĞƌƟĮ ĐĂͲƟŽŶƚĞƐƚƐĚĞǀ ĞůŽƉĞĚĂŶĚadministered by the UniversidadVeracruzana (UV) in the southeast-ern Mexican state of Veracruz. Thefirst suite was developed in the year 2000 by a small group of Englishlanguage teachers at the UV, as wellĂƐďǇƌĞƉƌĞƐĞŶƚĂƟǀ ĞƐĨƌŽŵƚŚĞ

ƌŝƟƐŚŽƵŶĐŝůĂŵďƌŝĚŐĞƐƐĞƐƐͲment, and Roehampton University’sCenter for Language Assessmentand Research (CLARe). The con-struct behind the EXAVER tests is tomeasure three language proficiency ůĞǀ ĞůƐŝĚĞŶƟĮ ĞĚŝŶƚŚĞŽƵŶĐŝůŽĨEurope's Common European Frame-work of Reference for Languages(CEFR), summarised in Table 1. The

EXAVER tests are administeredtwice a year at 11 language centersthroughout Veracruz.

Table 1: Levels of EXAVER tests andtheir corresponding CEFR Levels(adapted from Abad et al., 2011)

2.2 Test Structure

Each EXAVER test contains threeseparate papers. The structure ofeach paper is described below.

Table 2: EXAVER test structure;ĂŌĞƌƵŶŶĞϮϬϬϳ Ϳ

2.3 dĞƐƚůŽĐĂůŝǌĂƟŽŶ

According to O’Sullivan (2011), oneŽĨƚŚĞĚĞĮ ŶŝŶŐĐŚĂƌĂĐƚĞƌŝƐƟĐƐŽĨƚŚĞys ZĞǆĂŵŝŶĂƟŽŶƐŝƐƚŚĂƚƚŚĞǇƌĞƉƌĞƐĞŶƚ ƚŚĞĮ ƌƐƚƐǇƐƚĞŵĂƟĐĂƩ ĞŵƉƚƚŽĐƌĞĂƚĞĂůŽĐĂů Ăī ŽƌĚĂͲble, and sustainable language testsystem” (O’Sullivan, p.10). In focus-

EXAVER CEFR Council of Europe

1Upper Beginner

A2 Waystage

2Lower Intermediate

B1 Threshold

3Upper Intermediate

B2 Vantage

Paper 1ZĞĂĚŝŶŐĂŶĚt ƌŝƟŶŐ

Paper 2Listening

Paper 3Speaking

5 parts

Variety of tasks: matching,ŵƵůƟƉůĞĐŚŽŝĐĞŵŽĚŝĮ ĞĚcloze text

/ŶĚŝƌĞĐƚŵĞĂƐƵƌĞŽĨǁ ƌŝƟŶŐ

4 parts

Range from comprehension ofƌĞůĂƟǀ ĞůǇƐŚŽƌƚŝŶĨŽƌŵĂůĐŽŶǀ ĞƌͲƐĂƟŽŶƐƚŽĐŽŵƉƌĞŚĞŶƐŝŽŶŽĨŵŽƌĞĨŽƌŵĂůĂŶĚƐƵďƐƚĂŶƟĂůůǇůŽŶŐĞƌĐŽŶǀ ĞƌƐĂƟŽŶƐ

3 parts

Combine some type of inter-view task (interlocutor to candi-date), discussion task(between a pair of candidates)and a long-turn task(interlocutor to candidate)

30


Issue 63 30

ŝŶŐƚŚĞŝƌĂƩ ĞŶƟŽŶŽŶƚŚĞůŽĐĂůŐĞŽͲŐƌĂƉŚŝĐĐŽŶƚĞǆƚŽĨƚŚĞĞǆĂŵŝŶĂƟŽŶƐ(southeastern Mexico) and the par-ƟĐƵůĂƌŶĞĞĚƐŽĨƚŚĞĐĂŶĚŝĚĂƚĞƐǁ ŝƚŚͲin that context (students, primarily,of the Universidad Veracruzana),EXAVER’s test developers havehelped create a process now knownĂƐ ůŽĐĂůŝǌĂƟŽŶ K ^Ƶůůŝǀ ĂŶĚĞĮ ŶĞƐƚŚŝƐĂƐ ƚŚĞƉƌĂĐƟĐĞŽĨƚĂŬŝŶŐŝŶƚŽaccount those learning-focused fac-ƚŽƌƐƚŚĂƚĐĂŶŝŵƉĂĐƚŽŶůŝŶŐƵŝƐƟĐƉĞƌĨŽƌŵĂŶĐĞ ĂŶĚƚŚĞƌĞĐŽŐŶŝƟŽŶof the importance of test context ontest development…” (O’Sullivan,p.6).

Economic affordability was one of the first local variables that EXA-VER’s test developers considered.As the majority of EXAVER’s candi-ĚĂƚĞƐĐŽƵůĚŶŽƚ;ĂŶĚƐƟůůĐĂŶŶŽƚͿafford the cost of more reputable ŝŶƚĞƌŶĂƟŽŶĂůŶŐůŝƐŚůĂŶŐƵĂŐĞĐĞƌƟͲĮ ĐĂƟŽŶƚĞƐƚƐ ys Z ƐƚĞƐƚĚĞǀ ĞůͲopers decided to create a suite ofeconomically affordable tests, in line with median to lower incomebrackets based on the Mexican min-imum wage.2 Table 3 shows the cur-rent costs (as of September 2017) oftaking an EXAVER test, with approxi-mate equivalents in Euros.3

As of January 2017, the Mexicanminimum wage was approximately80 pesos per day.

For more details on the EXAVER ex-ĂŵŝŶĂƟŽŶƐĂŶĚƚŚĞys ZƚĞƐƚsystem, especially as they relate toůŽĐĂůŝǌĂƟŽŶ ƐĞĞďĂĚĞƚĂů“Developing affordable, ‘local’ tests: the EXAVER Project” in LanguagedĞƐƟŶŐdŚĞŽƌŝĞƐĂŶĚWƌĂĐƟĐĞƐ Ě Barry O’Sullivan (Palgrave Macmil-lan, 2011) pages 228-243.

dĂďůĞϯ ŽŵƉĂƌĂƟǀ ĞĐŽƐƚŽĨƚĂŬŝŶŐĂŶEXAVER test

3. Methodology

A mixed methods (quan → QUAL) ĂƉƉƌŽĂĐŚĨŽƌĚĂƚĂĐŽůůĞĐƟŽŶĂŶĚanalysis was used for the study. TheƋƵĂŶƟƚĂƟǀ ĞĚĂƚĂĐĂŵĞĨƌŽŵƚŚĞresponses of 245 EXAVER candi-dates who completed a web-basedƋƵĞƐƟŽŶŶĂŝƌĞǁ ŚŝĐŚǁ ĂƐĂĚŵŝŶŝƐͲtered in the summer of 2010 follow-ŝŶŐƚŚĞƐƉƌŝŶŐϮϬϭϬĂĚŵŝŶŝƐƚƌĂƟŽŶof EXAVER’s three levels. The ques-ƟŽŶŶĂŝƌĞŝŶĐůƵĚĞĚϰϮĐůŽƐĞĚ-format,ŵƵůƟƉůĞ-ĐŚŽŝĐĞƋƵĞƐƟŽŶƐ ĂŶĚƚǁ Žopen-ĞŶĚĞĚƋƵĞƐƟŽŶƐ K ĨƚŚĞclosed-ĨŽƌŵĂƚƋƵĞƐƟŽŶƐ ϭϬĞŵͲƉůŽǇĞĚĂ>ŝŬĞƌƚ ĐĂůĞǁ ŝƚŚŽƉƟŽŶƐspanning from 1 to 5, to ascertaincandidates' opinions about severaltopics related to the test. Excel Ver-sion 2003 was used to analyze theĚĂƚĂ dŚĞƋƵĂůŝƚĂƟǀ ĞĚĂƚĂĐĂŵĞfrom the author’s research journalfrom March to October 2010, andfrom semi-structured interviewsconducted in October 2010 withĨŽƵƌŽĨƚŚĞƋƵĞƐƟŽŶŶĂŝƌĞΖƐƌĞƐƉŽŶĚͲents.

4. Findings 4

Out of the 964 candidates who tookan EXAVER test in May 2010, 245 ofthem (or 25%) responded to theweb-based survey. Of these, 99 (orϰϬй ͿƟĐŬĞĚƚŚĞďŽǆĂƚƚŚĞĞŶĚŽĨthe survey, signifying their desire to

ƉĂƌƟĐŝƉĂƚĞŝŶĂƐĞŵŝ-structured in-ƚĞƌǀ ŝĞǁ dŚŝƐƌĞůĂƟǀ ĞůǇŚŝŐŚƌĂƚĞŽĨ

response was the first ŝŶĚŝĐĂƟŽŶŽĨƚŚĞĂƉƉĂƌͲent strong desire of EXA-VER’s candidates to havetheir voices heard.

Web-ďĂƐĞĚƋƵĞƐƟŽŶŶĂŝƌĞƐƵƌǀ ĞǇ

ƐƌĞƐĞĂƌĐŚŝŶƐƚƌƵŵĞŶƚƐ ƋƵĞƐƟŽŶͲŶĂŝƌĞƐŚĂǀ ĞƚŚĞŝƌĚŝƐƟŶĐƚĂĚͲvantages and disadvantages. WithƌĞŐĂƌĚƚŽƚŚĞůĂƩ ĞƌƌĞƐĞĂƌĐŚĞƌƐƐŽŵĞƟŵĞƐĐŽŵƉůĂŝŶĂďŽƵƚƚŚĞůĂĐŬŽĨĚĞƉƚŚĂŶĚƌŝĐŚŶĞƐƐŝŶŵƵůƟƉůĞ-choice responses (Dörnyei, 2007,p.115). For this reason, the re-searcher included two open-endedƋƵĞƐƟŽŶƐŝŶƚŚĞƐƵƌǀ ĞǇĂůŽŶŐǁ ŝƚŚƚŚĞϰϮŵƵůƟƉůĞ-ĐŚŽŝĐĞƋƵĞƐƟŽŶƐ While the responses to all of theƐƵƌǀ ĞǇƐƋƵĞƐƟŽŶƐƉƌŽǀ ŝĚĞĚŝŵͲportant feedback, the responses tothe two open-ĞŶĚĞĚƋƵĞƐƟŽŶƐ(numbers 17 and 30) are notewor-thy, due both to the high number ofcandidates who responded to them(well over half of the total 245 can-didates who took part in the sur-vey), as well as to the diversity oftheir answers. Summaries of theseresponses follow.5

Y ƵĞƐƟŽŶϭϳ “Do you feel that thereis anything we could include on theEXAVER website that might helpfuture candidates to feel less anx-ious and/or more confident before taking the test? If so, please writeyour comment(s) below, taking allthe space that is necessary.” Ques-ƟŽŶϭϳ ǇŝĞůĚĞĚϭϰϰƌĞƐƉŽŶƐĞƐ ŽƌŐĂͲnized into the following categories:

ϮϯƉŽƐŝƟǀ ĞƌĞƐƉŽŶƐĞƐ ƐƵĐŚĂƐ

LEVEL Cost in MX Pesos Cost in Euros

EXAVER 1 500 Approx. 23



31


Issue 63 31

“I didn’t hire a tutor or use anybooks to prepare for the test, as/ĨŽƵŶĚƚŚĞŝŶĨŽƌŵĂƟŽŶŽŶƚŚĞwebsite very useful.”

ϮϰŶĞŐĂƟǀ ĞƌĞƐƉŽŶƐĞƐ ƐƵĐŚĂƐ

dŚĞǁ ĂŝƟŶŐƟŵĞƚŽŐĞƚǇŽƵƌgrade is too long...you reallyneed to find a way to make it go faster.”

“I would have benefited from a greater variety, and greaterscale of difficulty, of test prep-ĂƌĂƟŽŶŵĂƚĞƌŝĂůƐ– the Sam-ple Tests on the website werereally easy and not very help-ful.”

ϵϳ ƐƵŐŐĞƐƟŽŶƐ ŶŽƚĂďůǇĨŽƌĨƵƌƚŚĞƌĚŽĐƵŵĞŶƚĂƟŽŶŽƌmaterials to include on theEXAVER website, such as:

a video of a sample speakingtest

a bibliography of literature toconsult to help prepare forthe test

ĂĚĞƐĐƌŝƉƟŽŶŽĨŚŽǁ ŐƌĂĚĞƐare calculated

Y ƵĞƐƟŽŶϯϬ“Do you have any oth-ĞƌĐŽŵŵĞŶƚƐ;ƉŽƐŝƟǀ ĞŽƌŶĞŐĂƟǀ ĞͿĂŶĚ ŽƌƐƵŐŐĞƐƟŽŶƐƚŚĂƚǇŽƵ ĚůŝŬĞƚŽadd regarding the specific test you ƚŽŽŬŽƌĂďŽƵƚƚŚĞys ZĞƌƟĮ ĐĂͲƟŽŶdĞƐƚƐŝŶŐĞŶĞƌĂů /ĨƐŽ ƉůĞĂƐĞwrite them below, taking all thespace that is necessary.” Y ƵĞƐƟŽŶ30 yielded 127 responses.

ϯϴƉŽƐŝƟǀ ĞƌĞƐƉŽŶƐĞƐ ƐƵĐŚĂƐ

“The EXAVER staff appeared to be very knowledgeable andwhen they gave the instruc-ƟŽŶƐŝŶŶŐůŝƐŚ ŝƚǁ ĂƐǀ ĞƌǇclear, which set me at easeand made me feel more confi-dent.”

ϲϭŶĞŐĂƟǀ ĞĐŽŵŵĞŶƚƐ ƐƵĐŚas:

t ŚŝůĞǁ ĂŝƟŶŐŝŶůŝŶĞƚŽĞŶͲter the test center, I was toldthat my name was not on thelist even though I had my reg-ŝƐƚƌĂƟŽŶƌĞĐĞŝƉƚ /ŶƚŚĞĞŶĚ/was able to take the test, butI felt very nervous.”

ϮϴƐƵŐŐĞƐƟŽŶƐ ƐƵĐŚĂƐ

“It would be nice to have amore detailed report on how Ifared in the test, such asknowing how I performed oneach part of the test, maybein terms of percentages.”

5. Discussion

5.1 Specific concerns

dŚĞƉŚƌĂƐŝŶŐŽĨY ƵĞƐƟŽŶϭϳ ǁ ŝƚŚspecial emphasis on the words“more confident” and “less anx-ŝŽƵƐ ǁ ĂƐŝŶƚĞŶƟŽŶĂůŝŶŽƌĚĞƌƚŽreflect the researcher’s premise that the less anxious and more confident candidates feel before taking a test,the more likely they are to performďĞƩĞƌ6 dŚĞƌĞůĂƟǀ ĞůǇůŽŶŐůŝƐƚŽĨƐƵŐŐĞƐƟŽŶƐ;ϵϳ ŝŶƚŽƚĂůͿƚŚĂƚĐĂŶĚŝͲdates gave in response to this ques-ƟŽŶŚĂǀ ĞƉƌŽǀ ĞŶƵƐĞĨƵůŝŶŚĞůƉŝŶŐEXAVER’s test developers improvethe quality of the website’s prepara-ƟŽŶŵĂƚĞƌŝĂůƐƐŽƚŚĂƚĐĂŶĚŝĚĂƚĞƐ

can indeed feel more confident and less anxious before taking a livetest.

Y ƵĞƐƟŽŶϯϬƐŚŽƵůĚƐĞĞŵĨĂŵŝůŝĂƌƚŽƋƵĂůŝƚĂƟǀ ĞƌĞƐĞĂƌĐŚĞƌƐ ƐŝŶĐĞŝƚŝƐthe classic “Do you have anythingelse to add?” that usually appears atthe end of an interview. It was con-sidered necessary as a ‘safety net’to ensure that candidates had theopportunity to state anything andeverything they wished to aboutpreparing for and taking an EXAVERtest.

KŶĞŽĨƚŚĞŶĞŐĂƟǀ ĞƌĞƐƉŽŶƐĞƐƚŽY ƵĞƐƟŽŶϯϬƌĞĨĞƌƌŝŶŐƚŽĂĐĂŶĚŝͲdate’s sense of anxiety over theirname not being found on the official list of test registrants, relates to theƚŚĞŵĞŽĨY ƵĞƐƟŽŶϭϳ /ƚƐŚŽƵůĚƌĞͲmind testers of the importance ofĂĚŽƉƟŶŐŵĞĂƐƵƌĞƐƚŚĂƚŚĞůƉƚŽavoid circumstances that can poten-ƟĂůůǇĐĂƵƐĞƵŶŶĞĐĞƐƐĂƌǇƐƚƌĞƐƐĂŶĚanxiety for candidates. Test examin-ers and administrators should devel-op a list of things that could possiblygo wrong on the day of the test,ŝĚĞŶƟĨǇŝŶŐĂŶĞī ĞĐƟǀ ĞƐŽůƵƟŽŶĨŽƌeach of them. Such a list shouldthen be printed and given to testinvigilators.

ǇĐŽŶƚƌĂƐƚ ŽŶĞŽĨƚŚĞƉŽƐŝƟǀ ĞƌĞͲƐƉŽŶƐĞƐƚŽY ƵĞƐƟŽŶϯϬŝůůƵƐƚƌĂƚĞƐŚŽǁ ĂƐĞĞŵŝŶŐůǇƌŽƵƟŶĞƚĂƐŬ(calmly and clearly reading the ini-ƟĂůŝŶƐƚƌƵĐƟŽŶƐŽŶĐĞĐĂŶĚŝĚĂƚĞƐĂƌĞseated) can actually serve to mini-mize test anxiety and boost candi-dates’ sense of confidence. Both examples reinforce the importanceŽĨĞŶƐƵƌŝŶŐƚŚĂƚƚŚĞƚĞƐƚƌĞĐĞƉƟŽŶ process (the way that candidates

32


Issue 63 32

are physically greeted and treatedby examiners and invigilators bothprior to and during the test) is assmooth and professional as possi-ble.

5.2 General concerns

Candidate responses to both theƋƵĞƐƟŽŶŶĂŝƌĞĂŶĚƚŚĞƐĞŵŝ-structured interviews provid-ĞĚĂƌŝĐŚƌĞƉƌĞƐĞŶƚĂƟŽŶŽĨƚŚĞdiversity of opinions, feelings,ƉĞƌĐĞƉƟŽŶƐ ĂŶĚĂƫ ƚƵĚĞƐthat EXAVER candidates haveabout the tests. They also pro-vided EXAVER’s test develop-ers with important insightsŝŶƚŽƐŽŵĞŽĨƚŚĞƉŽƐŝƟǀ ĞĂŶĚŶĞŐĂƟǀ ĞĐŽŶƐĞƋƵĞŶĐĞƐĨŽƌtest candidates as a result ofpreparing for and taking a lan-guage test. With regard to theƋƵĞƐƟŽŶŶĂŝƌĞƚŚĞƋƵĂŶƟƚǇand variety of responses bringto mind Shohamy’s (2001) ob-ƐĞƌǀ ĂƟŽŶƚŚĂƚƚŚĞŽǀ Ğƌǁ ŚĞůŵŝŶŐmajority of test candidates not onlyhave a strong need and desire toexpress their feelings about the testthey took, but they also have theinherent right to do so, and it is theresponsibility of language teachersand testers to enable them to do so(p.156). By providing for this, shefeels that testers can help democra-ƟǌĞƚŚĞĂĐƚŽĨƚĂŬŝŶŐĂƚĞƐƚƐŽƚŚĂƚthe experience becomes more of aĐŽůůĂďŽƌĂƟǀ ĞŚŽƌŝǌŽŶƚĂůƉƌŽĐĞƐƐ rather than an authoritarian, top-down one (Shohamy, p. 136-137).

It can be argued, however, that themost important step that takesplace in the overall process of solic-ŝƟŶŐĐĂŶĚŝĚĂƚĞĨĞĞĚďĂĐŬŝƐǁ ŚĂƚtesters finally end up using it for.

For this reason, one might correctlyrefer to the ‘final consequences’ of ĐŽŶƐĞƋƵĞŶƟĂůǀ ĂůŝĚŝƚǇdŚĞĮ ŶĂůĂĐͲƟŽŶƐƚĂŬĞŶďǇƚĞƐƚĚĞǀ ĞůŽƉĞƌƐŵĂǇvery well serve to increase the oc-ĐƵƌƌĞŶĐĞŽĨƉŽƐŝƟǀ ĞĐŽŶƐĞƋƵĞŶĐĞƐfor future candidates and to de-

ĐƌĞĂƐĞƚŚĞŽĐĐƵƌƌĞŶĐĞŽĨŶĞŐĂƟǀ Ğconsequences.

dŚĞĨŽůůŽǁ ŝŶŐĂĐƟŽŶƐ ďĂƐĞĚŽŶĐĂŶͲdidate feedback from this and otherstudies, have already been taken (orare currently being undertaken) byEXAVER administrators and serve toillustrate how a language test boardcan convert candidate feedback intoƉŽƐŝƟǀ ĞŝŵƉĂĐƚĨŽƌĨƵƚƵƌĞĐĂŶĚŝͲdates:

^ƚƌĞĂŵůŝŶĞĚƌĞŐŝƐƚƌĂƟŽŶƉƌŽĐĞƐƐ making it much easier for currentand future candidates to registerfor the tests

EĞǁ ŽŶůŝŶĞŐƌĂĚĞĂůůŽĐĂƟŽŶƉƌŽͲĐĞƐƐƚŽƐƵďƐƚĂŶƟĂůůǇƌĞĚƵĐĞƚŚĞǁ ĂŝƟŶŐƟŵĞĨŽƌƌĞĐĞŝǀ ŝŶŐŐƌĂĚĞƐ

Sample Speaking Test for each ofEXAVER’s three levels uploadedto the EXAVER website so thatƉŽƚĞŶƟĂůĐĂŶĚŝĚĂƚĞƐŚĂǀ ĞĂŶidea of the format of the test aswell as the type of tasks they canexpect (These tests complementƚŚĞƐĂŵƉůĞƌĞĂĚŝŶŐǁ ƌŝƟŶŐĂŶĚlistening tests on the websiteƐŝŶĐĞys Z ƐŝŶĐĞƉƟŽŶ Ϳ

ƌĂŌŝŶŐŽĨĂůŝƐƚŽĨĂĚŵŝŶŝƐƚƌĂͲƟǀ ĞƉƌŽĐĞĚƵƌĞƐƚŚĂƚŵĂǇďĞƉƌŽďůĞŵĂƟĐĨŽƌĞǆĂŵŝŶĞƌƐĂŶĚinvigilators on the test day, alongǁ ŝƚŚĐŽƌƌĞƐƉŽŶĚŝŶŐƐŽůƵƟŽŶƐ

ŝƐƐĞŵŝŶĂƟŽŶŽĨĐĂŶĚŝĚĂƚĞĨĞĞĚďĂĐŬƋƵĞƐƟŽŶŶĂŝƌĞƐĂƐĂǁ ĂǇŽĨĐŽŶƟŶƵŝŶŐƚŽŵŽŶŝƚŽƌƚŚĞƉŽƐͲiƟve and negaƟve consequencesfor candidates taking the tests

Analysis and discussion of ap-ƉƌŽƉƌŝĂƚĞĂĐƟŽŶ;ƐͿďĂƐĞĚŽŶĐĂŶͲdidate responses

Follow-through to confirm that ĂƉƉƌŽƉƌŝĂƚĞĂĐƟŽŶǁ ĂƐƚĂŬĞŶ

6. Conclusion

By now it has perhaps become ap-parent to the reader that what can-ĚŝĚĂƚĞĨĞĞĚďĂĐŬŝŶůĂŶŐƵĂŐĞƚĞƐƟŶŐactually relates to is a type of as-sessment that is more inclusive andĚĞŵŽĐƌĂƟĐŝŶŶĂƚƵƌĞƚŚĂŶƚŚĞƚƌĂĚŝͲƟŽŶĂůĂƵƚŚŽƌŝƚĂƌŝĂŶƚǇƉĞƚŚĂƚǁ ĂƐprevalent in so many assessmentcontexts throughout the world dur-ing much of the 20th century and,indeed, prior to that. 7

When test developers refuse to so-licit candidate feedback, or do sowithout following through on it, theƵŶĚĞŵŽĐƌĂƟĐŶĂƚƵƌĞŽĨƚŚĞĂƐƐĞƐƐͲment can be seen in terms of the

“With regard to the question-

naire, the quantity and variety

of responses bring to mind

Shohamy’s observation that

the overwhelming majority of

test candidates not only have

a strong need and desire to

express their feelings about the

test they took, but they also

have the inherent right to do

so, and it is the responsibility of

language teachers and testers

to enable them to do so.”

33


Issue 63 33

power and control that testers ex-ert over candidates. Conversely,when test developers solicit candi-ĚĂƚĞĨĞĞĚďĂĐŬĂŶĚƚĂŬĞƉŽƐŝƟǀ ĞĂĐƟŽŶƐďĂƐĞĚŽŶŝƚ ƚŚĞĚĞŵŽĐƌĂƟĐnature of the assessment is evi-denced as a horizontal and collabo-ƌĂƟǀ ĞƉƌŽĐĞƐƐ D ŽƌĞŽǀ ĞƌƚŚŝƐƉƌŽͲĐĞƐƐĞŶĐŽƵƌĂŐĞƐƚŚĞƉĂƌƟĐŝƉĂƟŽŶof not merely a few, but a wide va-riety of stakeholders, therebystrengthening even further theĚĞŵŽĐƌĂƟĐŶĂƚƵƌĞŽĨƚŚĞĂƐƐĞƐƐͲment.

Another important point that lan-guage test developers should con-sider when judging the validity oftheir assessments is that languageƚĞƐƟŶŐůŝŬĞĂŶǇƚǇƉĞŽĨƚĞƐƟŶŐŝƐ at best, an inexact science. Thereare many things that can go wrongon the day of a test which can in-terfere in its validity. The air condi-ƟŽŶŝŶŐŝŶĂŚŽƚĂŶĚŚƵŵŝĚƌŽŽŵcould stop working, forcing candi-dates to finish the test in uncom-ĨŽƌƚĂďůĞƉŚǇƐŝĐĂůĐŽŶĚŝƟŽŶƐ Ŷoral examiner could ask a candidateǁ ŚĂƚƚŚĞǇĚŝĚŽŶƚŚĞŝƌůĂƐƚǀ ĂĐĂƟŽŶwithout knowing that someone inthe candidate’s family died at thatƟŵĞ/ŶďŽƚŚŽĨƚŚĞƐĞŶŽƚŽǀ ĞƌůǇextraordinary cases, the candi-ĚĂƚĞƐĐŽŶĐĞŶƚƌĂƟŽŶĐŽƵůĚƉŽƐƐŝďůǇbe affected, thereby modifying his/her performance. This couldĐŽŶƐĞƋƵĞŶƚůǇŶĞŐĂƟǀ ĞůǇŝŵƉĂĐƚŽŶthe candidate’s score and provide aĨĂůƐĞƌĞŇĞĐƟŽŶŽĨŚŝƐŽƌŚĞƌƚƌƵĞability.

The above examples represent realƐŝƚƵĂƟŽŶƐƚŚĂƚŚĂǀ ĞƚĂŬĞŶƉůĂĐĞduring real EXAVER test administra-ƟŽŶƐ ƐůĂŶŐƵĂŐĞƚĞƐƚĞƌƐǁ ŽƌŬ

with real people in the real world,ƌĞĂů;ĂŶĚŽŌĞŶƟŵĞƐƵŶĨŽƌĞƐĞĞĂďůĞor uncontrollable) problems areůŝŬĞůǇƚŽĐŽŶƟŶƵĞƚŽŽĐĐƵƌ, Žǁ Ğǀ Ͳer, when it comes to designing andadministering tests, there are manythings that testers can indeed con-trol, including:

Concern for the test’s most im-portant stakeholder: the candi-date

The psychometric quality of thetest, i.e. its validity and reliability

ŽůůĞĐƟǀ ĞĞůĞŵĞŶƚƐŽĨƚŚĞƚĞƐƚƐǇƐƚĞŵŽƌƚŚĞŽƉĞƌĂƟŽŶƐƚŚĂƚare external to the test per se,ƐƵĐŚĂƐƚŚĞƚĞƐƚƌĞŐŝƐƚƌĂƟŽŶ ŽƌŝͲĞŶƚĂƟŽŶ ĂŶĚƉƌĞƉĂƌĂƟŽŶƉƌŽͲcesses and the way that candi-dates are treated by examinersand invigilators on the day of thetest

ĞŝŶŐƌĞƐƉŽŶƐŝďůĞĂŶĚĞī ĞĐƟǀ Ğexaminers, e.g. giving fair andnon-ƉĂƌƟĂůƚƌĞĂƚŵĞŶƚƚŽĂůůĐĂŶͲdidates and following-up with a‘post-exam’ report with a list ofthings that went right and wrongduring the test

By concerning themselves withthese and other important

variables, language test-ers can help safeguardthe overall fairness andintegrity of the test andthe test system. In sodoing, they also help tounderscore the differ-ence between assess-ments that, on the onehand, are moving to-wards a more dynamic,responsible and demo-ĐƌĂƟĐŵŽĚĞůĂŶĚŽŶƚŚĞother hand, ones thatĐŽŶƟŶƵĞƚŽƌĞŵĂŝŶmore stagnant and con-ǀ ĞŶƟŽŶĂůŝŶŶĂƚƵƌĞ

References provided on request

Footnotes:

1. dƌĂŶƐůĂƟŽŶ /ŶŵǇŚƵŵďůĞŽƉŝŶͲŝŽŶ dŚĞƉƌĞƐĞŶƚĂƌƟĐůĞŝƐĂŶabridged version of “Consider thecandidate: using test-taker feed-back to enhance quality and validi-ƚǇŝŶůĂŶŐƵĂŐĞƚĞƐƟŶŐ ƉƵďůŝƐŚĞĚin e-TEALS: An e-journal of TeacherĚƵĐĂƟŽŶĂŶĚƉƉůŝĞĚ>ĂŶŐƵĂŐĞStudies 5 (2014): 1-23. ISSN 1647-ϳ ϭϮy ŚƩ Ɖ ůĞƌůĞƚƌĂƐ ƵƉ Ɖƚuploads/ficheiros/13086.pdf

“When test developers refuse to

solicit candidate feedback, or

do so without following through

on it, the undemocratic nature

of the assessment can be seen in

terms of the power and control

that testers exert over candi-

dates. Conversely, when test de-

velopers solicit candidate feed-

back and take positive actions

based on it, the democratic na-

ture of the assessment is evi-

denced as a horizontal and col-

laborative process.”

34


Issue 63 34

2. As of January 2017, the Mexicanminimum wage was approximately80 pesos per day.

3. For more details on the EXAVER ex-ĂŵŝŶĂƟŽŶƐĂŶĚƚŚĞys ZƚĞƐƚsystem, especially as they relate toůŽĐĂůŝǌĂƟŽŶ ƐĞĞďĂĚĞƚĂů“Developing affordable, ‘local’ tests: the EXAVER Project” in LanguagedĞƐƟŶŐdŚĞŽƌŝĞƐĂŶĚWƌĂĐƟĐĞƐ Ě Barry O’Sullivan (Palgrave Macmil-lan, 2011) pages 228-243.

4. ƵĞƚŽƐƉĂĐĞĐŽŶƐŝĚĞƌĂƟŽŶƐ Į ŶĚͲŝŶŐƐƌĞůĂƚĞĚƚŽƚŚĞƋƵĂůŝƚĂƟǀ ĞƉŚĂƐĞof the study could not be includedbut are available from the author [email protected]

5. The web-ďĂƐĞĚƋƵĞƐƟŽŶŶĂŝƌĞŽƌŝŐŝͲnally appeared in Spanish and wassubsequently translated into Eng-lish. Candidate responses to theƋƵĞƐƟŽŶŶĂŝƌĞĨŽůůŽǁ ĞĚƚŚŝƐƐĂŵĞƉĂƩ ĞƌŶ

6. While it could be argued that thispremise is based on common sense,it actually mirrored Bachman and

Palmer’s similar hypothesis (1996,p.32).

7. dŚĞƚƌĂĚŝƟŽŶĂůŽƌĂƵƚŚŽƌŝƚĂƌŝĂŶŵŽĚĞůŽĨĞĚƵĐĂƟŽŶĂŶĚĂƐƐĞƐƐŵĞŶƚŝƐƐƟůůƉƌĞǀ ĂůĞŶƚŝŶŵĂŶǇƉĂƌƚƐŽĨƚŚĞworld today, including in many edu-ĐĂƟŽŶĂůĐŽŶƚĞǆƚƐŝŶD ĞǆŝĐŽ

35


Issue 63 35

You are responsible for developing an inter-

mediate level test for your school. There are

ten intermediate classes with ten different

teachers. They all use different course

books. If the students do well enough, they

can move to the next level. Teachers don’t

have a lot of time for marking. Tests will be

run monthly and there will be 150 students

taking the test.

Possible approaches/considerations:

- Collect course books used and analyse lan-

guage, functions, skills etc. used in each book.

- Identify common crossovers.

- Evaluate whether the test should include any

skills element, taking timing and resources into

account.

- Consider that you may need multiple versions.

- Pilot tests with the teachers (ask them to com-

plete within the time etc).

- For practicality, you could assess the function-

ing after the first tests (at a most basic level, ask

teachers to name the students in class that are

the strongest and weakest, and compare

against results).

- Changes can be made after testing to further

improve test functioning, possibly based on

placement of students and how well this is work-

ing.

You are designing a business English test

for telesales workers. You will be responsi-

ble for giving the test to new workers

(roughly 5 workers a month), and with the

results they will be given specific modules

to improve their working capability in Eng-

lish. All new employees already have a B2

level of English.

Possible approaches/considerations:

- Analyse the language, functions and skills that

are used in the role by studying existing work-

ers. Are they using a script? Do they go off

script?

- Consider the impact carefully; could employ-

ees lose their job due to this test? Do they know

or not know this? Do you know this?

- Practicality is less of an issue compared to

scenario 1, and you may be able to activate the

test language in authentic tasks that they will

carry out in their work.

- Ensure test design has clear links to the mod-

ules that you can recommend.

- Consider carefully when the tests will be done;

there could be some interactivity here between

their training for the job and their test perfor-

mance.

- Pilot the test on existing employees; they could

reap the benefits of free further training ideas!

IATEFL TEASIG WebinarDeveloping a test: Where do you start? When should you stop?

24 October 2017Jo Tomlinson

As we didn’t get time to discuss the scenarios at the end of the webinar, Jo has provided us witha small collection of thoughts and considera-tions. She is sure there are many more.

36


Issue 63 36

The unanswered questions asked during the

course of the webinar have been answered by

Jo in the following:

Is a test always a compromise governed by

rules?

I would say so. From my experience, test devel-

opment and the resulting test is always a series

of compromises. Test development often re-

quires two (possibly) distinct approaches. The

first is understanding and using good practice

standards. The second is pragmatism. Put simp-

ly, it’s the compromise between what should be

done and what can be done. I’m sure all of us

testers have an ideal way of testing in a specific

situation, but available resources often limit that.

We just have to do the best we can with what

we’ve got!

Isn't test development more cyclical than

linear? Or like a jigsaw puzzle where the

pieces never manage to fit together?

I think it definitely can be. Especially if the test is

to be repeated (perhaps with subsequent ver-

sions) over time. If there is a single test being

created, I personally like the stair metaphor as it

gives an idea of moving onwards (and up-

wards!), yet we must acknowledge and embrace

the fact that we’re bound to move up and down

quite a few stairs before getting to the top.

I also like the jigsaw puzzle idea. Perhaps our

aim should be to fit enough pieces together to

be able to ‘see the picture’!

How important is it that you speak to all

stakeholders in their own language – learn-

ers, school head, HR, bosses, parents?

When creating a test, I see no reason why we

shouldn’t do this if we can. The more we know

from everybody involved, the more we can un-

derstand the context. Almost nobody likes any-

thing that is forced upon them. The more we val-

ue the opinions of those involved, and express

this in a dialogue, the more likely they are to val-

ue the end product.

How much value do you place on ancillary

parts of test development such as item writ-

ing, rater training, statistical analysis, and

pilot feedback?

These areas are fundamental to test develop-

ment, but the part they play depends on the test

design and context. However, if your test design

includes these elements and they aren’t carried

out well enough, then you can’t actualise that

particular test design. Take, for example, criteria

that raters don’t particularly understand or are

self-conflicting; it doesn’t matter if you have the

best plans in the world, the results from that test

won’t be able to tell you much.

What type of test do you personally like con-

structing best?

This is a difficult question because I enjoy con-

structing all tests (honestly!). I would say I par-

ticularly enjoy analysing the construct. I work

with a colleague who comes from a linguistics

background in contrast to my testing back-

ground, so you can imagine the conversations. I

particularly enjoy work that gives us the creative

means to try something new. Integrated tests

are interesting to work on, simply because they

can be quite challenging from a development

angle.

Can you tell us something more about note-

taking as a task? Is note-taking a fair way of

testing these days, as we write so little? Or

could we do it on a tablet?

Let’s look at this from an EAP perspective. Note

-taking still exists, of course, and might be called

the most authentic (yet messy) way of testing

listening. There is a lot that can go wrong with

note-taking as a testing tool, especially in the

37


Issue 63 37

rating, yet there are ways in which these prob-

lems can be limited.

However, as we discussed in the webinar, note-

taking in general is moving on. More and more

students are recording their lectures and taking

photos of the slides. Now, I suppose an essen-

tial question for us as test developers is ‘What

are those students doing with that information

after the lecture?’

Good point regarding the way different ex-

aminers interpret particular assessment cri-

teria (e.g. spelling). How should we deal with

this and make sure that different assessors

interpret the assessment criteria they're sup-

posed to be using in the same way? Regular

monitoring and annual (re-)training sessions

don't seem to help a lot as the grades are

still different when we do random monitor-

ing.

Yes, I think it’s always a somewhat constant

struggle. If we take spelling into consideration,

then from my experience allowing degrees of

flexibility in spelling can be extremely problemat-

ic as what is an acceptable error for one rater

will be unacceptable for another. There are also

numerous problems with having a list of

‘acceptable spelling errors’. In this instance, I

would be tough and say no deviation from the

correct spelling is allowed. I also explain why

and ask the raters whether they think it is fair

that one student should ‘pass’ while another

should ‘fail’ due to a generous rater giving a few

more marks and a tougher rater giving a few

less. Most agree that it’s not acceptable.

In terms of wider interpretation of the marking

criteria, I think it is essential to discuss the fun-

damental terms for which we perhaps take for

granted that we all have a shared meaning (for

example, ‘task achievement’ or ‘cohesion’), as

well as analyse and discuss degrees of achieve-

ment and what they mean (e.g. ‘convincing’,

‘minimal’ etc.) and show examples of these.

Timing is also important; ensure the training/

standardisation is fresh in raters’ heads wherev-

er possible, and perhaps double mark and feed-

back (depending on whether this is feasible).

How much space do you think there is for

creativity in testing?

I hope there is a lot of space for creativity. The

more we can facilitate testing that is learner and

context specific, the more meaningful it can be

for the test-takers and also in terms of results

within the particular context. I think that one of

our aims as test developers (and perhaps this is

sometimes understated) is to assess in a way

that candidates find a test a joy to do, rather

than a hurdle to overcome.

38


Issue 63 38

Where are you based?Dubai, UAE

How long have you been there? / Which othercountries have you worked in?I’ve been in the Gulf for 25 years and have worked inthe US, France, Oman and the UAE.

How long have you been a member of IATEFL?As far back as I can remember.

Why did you choose TEASIG as your SIG?D ǇƉƌŝŵĂƌǇĂƌĞĂŽĨƐƉĞĐŝĂůŝǌĂƟŽŶŝƐƚĞƐƟŶŐassessment.

Are you a member of any other SIGs?No not with IATEFL but I chair two SIGs with ind^K>ƌĂďŝĂ ƚŚĞdĞƐƟŶŐƐƐĞƐƐŵĞŶƚĂŶĚǀ ĂůƵĂͲƟŽŶ /' ĂƐǁ ĞůůĂƐƚŚĞ>ĞĂĚĞƌƐŚŝƉĂŶĚD ĂŶĂŐĞŵĞŶƚSIG.

What do you get out of TEASIG?TEASIG is a community within IATEFL where every-one has the same or similar interests; I enjoy thisĚǇŶĂŵŝĐĂŶĚĞƐƉĞĐŝĂůůǇŵĞĞƟŶŐƚĞƐƚĞƌƐĨƌŽŵĂƌŽƵŶĚthe world and learning about what they are doingŶŽƚŽŶůǇŝŶƚŚĞĮ ĞůĚŽĨ>dďƵƚŝŶůĂŶŐƵĂŐĞƚĞƐƟŶŐand assessment.

t ŚĂƚŽƚŚĞƌƐĞƌǀ ŝĐĞƐŽƌĂĐƟǀ ŝƟĞƐǁ ŽƵůĚǇŽƵǁ ĞůͲcome?I would welcome more peer-edited and reviewedƉƵďůŝĐĂƟŽŶƐĨƌŽŵŽƵƌĐŽŵŵƵŶŝƚǇ

What sort of work do you do?/ĂŵĂĨƵůůƟŵĞĨĂĐƵůƚǇŵĞŵďĞƌĂƚƵďĂŝD ĞŶ ƐŽůͲlege where I teach General Studies courses.

, Žǁ ŵƵĐŚŽĨǇŽƵƌǁ ŽƌŬƟŵĞŝƐƐƉĞŶƚŽŶƚĞƐƟŶŐĞǀ ĂůƵĂƟŽŶĂŶĚĂƐƐĞƐƐŵĞŶƚ In my job about 15%ŽĨŵǇƟŵĞŝƐƐƉĞŶƚŽŶĂƐƐĞƐƐŵĞŶƚŽƌĂƐƐĞƐƐŵĞŶƚƌĞůĂƚĞĚĂĐƟǀ ŝƟĞƐ ŝŶŵǇƟŵĞŽƵƚƐŝĚĞƚŚĞǁ ŽƌŬƉůĂĐĞ/ƐƉĞŶĂĂŐƌĞĂƚĚĞĂůŽĨƟŵĞƚƌĂŝŶŝŶŐƚĞĂĐŚĞƌƐŚŽǁ ƚŽǁ ƌŝƚĞƚĞƐƚƐĂŶĚǁ ƌŝƟŶŐďŽŽŬƐŽŶĂƐƐĞƐƐŵĞŶƚ

Do you see yourself asmore of a teacher oras more of a tester?I’d like to say ‘both’but in actual fact I amnow spending more ofŵǇƟŵĞĂƐĂƚĞĂĐŚĞƌ

Do you work in a teamor alone?I believe that team-work is crucially im-portant so I work inteams much more sothan alone.

Do you set tests?zĞƐ /ĚŽďƵƚŵŽƌĞŽĨŵǇƚĞƐƟŶŐƟŵĞŝƐƐƉĞŶƚĞĚͲŝƟŶŐƚĞƐƚƐƚŚĂƚŽƚŚĞƌƐŚĂǀ Ğǁ ƌŝƩ ĞŶĂŶĚĂĚǀ ŝƐŝŶŐŽŶƚĞƐƟŶŐĂŶĚĂƐƐĞƐƐŵĞŶƚŵĂƩ ĞƌƐ

What sort of tests are they? How high are thestakes for the test-takers?They are usually midterm or final assessments of ďŽƚŚƚŚĞĨŽƌŵĂƟǀ ĞĂŶĚƚŚĞƐƵŵŵĂƟǀ ĞŬŝŶĚ ƚŚĞůĂƩ ĞƌĂƌĞƉƌĞƩǇŚŝŐŚƐƚĂŬĞƐĂƐƚŚĞǇƵƐƵĂůůǇĐŽƵŶƚĨŽƌ30% of a student’s final grade.

Are the tests you set published in any form?Not any more although I worked for many years asthe Chief Examiner for the English B component ofƚŚĞ/ŶƚĞƌŶĂƟŽŶĂůĂĐĐĂůĂƵƌĞĂƚĞĂŶĚŵĂŶǇŽĨƚŚŽƐĞtests were published.

ŽǇŽƵĞŶũŽǇƐĞƫ ŶŐƚĞƐƚƐzƵƉ ĂŌĞƌƚƌĂǀ ĞůůŝŶŐǁ ĂƚĐŚŝŶŐ' ĂŵĞŽĨdŚƌŽŶĞƐĂŶĚscuba diving, it’s my favorite thing to do!

Can you use the same set of skills and principles inseveral different contexts? I believe so. I feel the way we package ourselvesand this includes our skills and principles is very im-portant.

TEASIG Member Spotlight: Dr Christine Coombe.

39


Issue 63 39

ŽǇŽƵĐĂƌƌǇŽƵƚƌĞƐĞĂƌĐŚŽŶƚĞƐƟŶŐĂŶĚƌĞůĂƚĞĚfields?Yes.

, Ăǀ ĞǇŽƵƉƵďůŝƐŚĞĚŽŶƚĞƐƟŶŐŝƐƐƵĞƐ Yes, I have a few research-ďĂƐĞĚĂƌƟĐůĞƐŝŶƚŽƉƟĞƌũŽƵƌŶĂůƐůŝŬĞ>ĂŶŐƵĂŐĞdĞƐƟŶŐd^K>Y ƵĂƌƚĞƌůǇĂŶĚLanguage Teaching. However, my favorite kind ofpublishing is on helping teachers increase their as-sessment literacy.

, Ăǀ ĞǇŽƵƐƉŽŬĞŶĂƚĐŽŶĨĞƌĞŶĐĞƐŽŶƚĞƐƟŶŐŝƐƐƵĞƐYes, lots.

ŽǇŽƵĚŝƐĐƵƐƐƚĞƐƟŶŐŝƐƐƵĞƐǁ ŝƚŚĐŽůůĞĂŐƵĞƐ D ǇĐŽůůĞĂŐƵĞƐŽŌĞŶĐŽŶƐƵůƚŵĞŽŶƚĞƐƟŶŐŝƐƐƵĞƐ

Do you feel qualified to set tests? Yes very much so.

/ĨǇŽƵĚŽŶŽƚƐĞƚƐĞƚƐ ǁ ŚĂƚƚĞƐƟŶŐ-related work doyou do (advisory, marking, curriculum design …)?

/ƉƌĞƩǇŵƵĐŚĚŽŝƚĂůůƚĞƐƚƐĞƫ ŶŐĂĚǀ ŝƐŝŶŐŽŶƚĞƐƚƐ marking, examining etc.

ŚƌŝƐƟŶĞƚŚĂŶŬǇŽƵǀ ĞƌǇŵƵĐŚ

Email: [email protected]

40


Issue 63 40

Where are you based?I´m based in Brasília, the capital of Brazil, right in thecentre of the country.

How long have you been there?For almost five years.

Which other countries have you worked in?I´ve worked only in Brazil.

How long have you been a member of IATEFL?For about 4 years I think.

Why did you choose TEASIG as your SIG?ĞĐĂƵƐĞ/ƚŚŝŶŬƚĞƐƟŶŐĞǀ ĂůƵĂƟŽŶĂŶĚĂƐƐĞƐƐŵĞŶƚare crucial issues for any language teacher to knowabout.

Are you a member of any other SIGs?No, I´m not.

What do you get out of TEASIG?The most important thing, in my opinion, is thatTEASIG allows me to be in contact with people thatshare the same interests as mine – this allows me tolearn from other people, exchange ideas with peo-ple from different parts of the world and find solu-ƟŽŶƐƚŽĐŽŵŵŽŶƉƌŽďůĞŵƐǁ ĞŵŝŐŚƚŚĂǀ ĞŝŶŽƵƌdaily work.

t ŚĂƚŽƚŚĞƌƐĞƌǀ ŝĐĞƐŽƌĂĐƟǀ ŝƟĞƐǁ ŽƵůĚǇŽƵǁ ĞůͲcome?As I don´t live in Europe, I would very much welcomeŚĂǀ ŝŶŐŵŽƌĞŝŶĨŽƌŵĂƟŽŶŽŶǁ ŚĂƚǁ ĂƐƉƌĞƐĞŶƚĞĚand discussed in the events promoted by thed^/' ƉĂƌƟĐƵůĂƌůǇƚŚĞŽŶĞ-day ones!

What sort of work do you do?I am an English language teacher educator and Iteach both undergraduate (future English teachers)and MA students (most of them teachers working inpublic schools).

, Žǁ ŵƵĐŚŽĨǇŽƵƌǁ ŽƌŬƟŵĞŝƐƐƉĞŶƚŽŶƚĞƐƟŶŐ

Ğǀ ĂůƵĂƟŽŶĂŶĚĂƐͲsessment?A lot, not only as ateacher but also as aresearcher and as anMA tutor.

Do you see yourselfas more of a teacheror as more of a test-er?As a teacher, for sure.

Do you work in a team or alone?Mostly alone, unfortunately.

Do you set tests?Yes, I do.

What sort of tests are they?D ŽƐƚŽĨƚŚĞƟŵĞƚŚĞǇĂƌĞĐůĂƐƐƌŽŽŵƚĞƐƚƐ ďƵƚ/ĂŵĂůƐŽƉĂƌƚŽĨƚŚĞĐŽŵŵŝƩ ĞĞƚŚĂƚƐĞƚƐĞŶƚƌĂŶĐĞĞǆĂŵƐfor the post-ŐƌĂĚƵĂƟŽŶƉƌŽŐƌĂŵŵĞŝŶƉƉůŝĞĚ>ŝŶͲŐƵŝƐƟĐƐŽĨŵǇĚĞƉĂƌƚŵĞŶƚ

How high are the stakes for the test-takers?They are high states because they are related to thestudent´s progress at university – in the case of theĐůĂƐƐƌŽŽŵƚĞƐƚƐ ĂŶĚƚŚĞƐƵĐĐĞƐƐŽƌŶŽƚŝŶƐƚĂƌƟŶŐĂŶMA.

Are the tests you set published in any form?No, they´re for our own use.

ŽǇŽƵĞŶũŽǇƐĞƫ ŶŐƚĞƐƚƐ Yes, a lot!

Can you use the same set of skills and principles inseveral different contexts? Yes, sure, as long as you know the basic principles ofassessment.

ŽǇŽƵĐĂƌƌǇŽƵƚƌĞƐĞĂƌĐŚŽŶƚĞƐƟŶŐĂŶĚƌĞůĂƚĞĚfields?

TEASIG Member Spotlight: Gladys Quevedo Camargo.

41


Issue 63 41

Yes. At the moment I´m carrying out some researchon assessment literacy in Brazil for my post-docstudies.

, Ăǀ ĞǇŽƵƉƵďůŝƐŚĞĚŽŶƚĞƐƟŶŐŝƐƐƵĞƐ zĞƐ /ǲǀ ĞŐŽƚƐŽŵĞĂƌƟĐůĞƐƉƵďůŝƐŚĞĚŝŶƌĂǌŝůĂŶĚone abroad.

, Ăǀ ĞǇŽƵƐƉŽŬĞŶĂƚĐŽŶĨĞƌĞŶĐĞƐŽŶƚĞƐƟŶŐŝƐƐƵĞƐ Yes, I have.

ŽǇŽƵĚŝƐĐƵƐƐƚĞƐƟŶŐŝƐƐƵĞƐǁ ŝƚŚĐŽůůĞĂŐƵĞƐ Yes, a lot and whenever I have the opportunity!

Do you feel qualified to set tests? Yes, but assessment is a very broad and complexarea, so there´s always something to learn.

Gladys, thank you very much!

Email: [email protected]

.

42


Issue 63 42

ŽƉǇƌŝŐŚƚE ŽƟĐĞ

/d&>ƌĞƚĂŝŶƐƚŚĞƌŝŐŚƚƚŽƌĞƉƌŽĚƵĐĞƉĂƌƚŽƌĂůůŽĨƚŚŝƐƉƵďůŝĐĂƟŽŶŝŶŽƚŚĞƌƉƵďůŝĐĂƟŽŶƐ ŝŶĐůƵĚŝŶŐƌĞƚĂŝůĞĚŝƟŽŶƐŽŶƚƌŝďƵƟŽŶƐƚŽƚŚŝƐƉƵďůŝĐĂƟŽŶƌĞŵĂŝŶƚŚĞŝŶƚĞůůĞĐƚƵĂůƉƌŽƉĞƌƚǇŽĨƚŚĞĂƵƚŚŽƌƐ ŶǇrequests toƌĞƉƌŽĚƵĐĞĂƉĂƌƟĐƵůĂƌĂƌƟĐůĞƐŚŽƵůĚďĞƐĞŶƚƚŽƚŚĞƌĞůĞǀ ĂŶƚĐŽŶƚƌŝďƵƚŽƌĂŶĚŶŽƚ/d&>ƌƟĐůĞƐǁ ŚŝĐŚŚĂǀ ĞĮ ƌƐƚĂƉƉĞĂƌĞĚŝŶ/d&>ƉƵďůŝĐĂƟŽŶƐŵƵƐƚĂĐŬŶŽǁ ůĞĚŐĞƚŚĞ/d&>ƉƵďůŝĐĂƟŽŶĂƐƚŚĞŽƌŝŐŝŶĂůƐŽƵƌĐĞŽĨƚŚĞĂƌƟĐůĞŝĨƌĞƉƌŝŶƚĞĚĞůƐĞǁ ŚĞƌĞ++++++++++++++++++++++++++++++++++++++++++++++++++DisclaimerViews expressed in this IATEFL—those of the editor(s), of IATEFL TEASIG, of IATEFL or its staff or trustees.

Who is IATEFL for?

IATEFL is truly international with the majority of our members living in countries whereEnglish is not the first language. Members contribute greatly to the life of the Associationin a wide variety of ways: through participation in our 16 Special Interest Groups (SIGs);contributions to any of our magazines: IATEFL Voices and SIG publications; volunteeringon IATEFL committees; communicating developmental ideas to Head Office and giving pa-pers at, or simply attending our many conferences, workshops and seminars. The Associa-tion is managed and administered from an office in Faversham, UK. The nine remuneratedstaff are supported by volunteer committees, including the IATEFL Trustees and the Spe-cial Interest Group committees consisting of ELT professionals from a range of countries.

The aims of IATEFL are to:

· Benefit English language teachers all over the world providing opportunities forpersonal and professional development.

· Enable the international network of ELT professionals to grow, by encouraging andfostering the regional and local groupings, so that members can learn from eachother.

· Encourage grassroots professionalism where all categories of members at whateverstage of their career can make significant contributions and continue to learn.

Linking, developing & supporting ELT professionals worldwideIATEFL Patron: Professor David Crystal

Newsletter Issue 63 (December 2017) are not necessarily

newsletter december 2017 - universidad veracruzana · 40 teasig member spotlight gladys quevedo...

Documents