challenge to the assessment of speaking (derince cakmak)
TRANSCRIPT
A Challenging Dimension to the
Assessment of Speaking: CEFR through ELF
Zeynep Mine Derince & Pelin Tekinalp Çakmak
EAQUALS International Conference
Budapest
25-26 April 2014
Outline
Integration of CEFR into the preparatory school curricula in
Turkey
Marmara University School of Foreign Languages
Before CEFR
After CEFR
Implementation of CEFR--In-service Training Sessions
ELF training workshop
Suggestions
Preparatory Schools in Turkey
One year intensive program
Prep Schools consist of students of various departments: medicine, engineering, social sciences, communication, tourism etc.
Prep students study in:
100 % English-medium departments
30 % English-medium departments
Proficiency levels of students vary at the beginning
Preparatory Schools in Turkey
Foundation and state prep schools
Number of preparatory schools: 150
Number of English instructors in each school: 50-150
Number of preparatory students in each school: 500-2500
Schools have testing offices, curriculum units, TDUs.
A survey for Prep Schools
30 schools were included.
Questions on missions, demographic info, practices.
11 out of 20 state schools have started using CEFR (only
half), however, 9 out of 10 foundation schools (all but one)
have employed CEFR.
Mission statements of
School of Foreign Languages
Which of the following sections are
included in the exams specified below?
0
5
10
15
20
25
30
35
Grammar Speaking Reading Listening Writing
Placement
Proficiency
Achievement
What does the graph display?
These schools claim that they have adopted a CEFR-based
curriculum, however ,
It is interesting to see in the results that grammar rates as
much as the reading and writing:
22 schools out of 30 ask grammar explicitly in the
proficiency, 27 in the achievement exam and 29 in the
placement exam.
Speaking is not tested at all in half of these schools.
Before CEFR
Curriculum
Elementary, Pre-intermediate and Intermediate levels
Grammar, Listening, Reading, Writing
Skills disintegrated
Assessment
1st achievement- Grammar and Listening
2nd achievement –Reading and Writing
3rd achievement- Grammar and Listening
4th achievement –Reading and Writing
Proficiency- Grammar-Listening-Reading- Writing-(Grammar made up 50% of the final score.)
After CEFR Curriculum
3 levels: A1-A2-B1
4 skills integrated instruction
Assessment
1st Progress Test- Reading, Writing, Listening & Speaking
2nd Progress Test- Reading, Writing, Listening & Speaking
3rd Progress Test- Reading, Writing, Listening & Speaking
4th Progress Test- Reading, Writing, Listening & Speaking
Proficiency- Reading, Writing, Listening & Speaking
All 4 Skills are assessed out of 25 points.
Before the 1st Progress Exam
Two analytical rubrics prepared by the curriculum team: A
rubric for basic users /A rubric for independent users
A short workshop on rubrics
Purpose of the study
1st progress exam: speaking skill scores are lower that the
other three skill scores.
Focused on two factors: Being familiar with the illustrative
scales of CEFR and English as a Lingua Franca.
The goal of this study is to find out if familiarization with the
CEFR and ELF enables raters to assign reliable scores in line
with other skills assessed in the test.
In-service Training
Materials
Relating Language Examinations to the Common European
Framework of Reference for Languages: Learning, Teaching,
Assessment (CEFR) –MANUAL
CEFTrain (www.CEFtrain.net)
Video samples of local learners
Workshops on ELF, native vs. non-native speakers
In-service Training- Session 1
Before the session
Section 3.6 in the CEFR (English pp. 3336) that describes
the salient features of the levels.
During the session
Sorting the text for the different levels in Table A1-The task is
to assign the descriptors to levels A1C2.
After the initial input on the CEFR,
Sorting the individual descriptors from a CEFR scale
More in-depth work and discussion of CEFR levels in
relation to the descriptors for speaking
In-service Training- Session 2
Before the session
The website CEFTrain (www.CEFtrain.net) -a hands-on
example.
During the session
The aim is to provide participants with sufficient awareness
of the CEFR levels to analyse and assess test tasks and
performances in relation to the CEFR levels.
In-service Training- Session 3
Before the session
Analytic rubrics for scoring
During the session
Rating video samples of local learners: non-native to non-native
interaction
In-service Training- Session 4:
English as a Lingua Franca
ELF has emerged as a way of referring to
communication of English between speakers with
different first languages, across lingua cultural
boundaries. ( House, 1999; Seidlhofer, 2002, 2005)
English is being shaped at least as much by its non-
native speakers as by its native speakers
For the majority of its users English is a Foreign
language, yet the vast majority of verbal exchanges in
English do not involve any native speakers.
In-service Training- Session 4:
English as a Lingua Franca
Global use of English is on the rise
Questioning the native speakers’ long accepted ownership of English (Widdowson, 1994)
Native speaker norms act as a powerful gate-keeping device which has little to do with intelligibility but a great deal with socio-economic factors.
Non-native speakers as language users in their own right (Cook,1999).
The importance of respecting and building on local values, beliefs and ways of doing things (Holliday, 1994).
Kachru’s circles
In-service Training- Session 4:
English as a Lingua Franca
Some language testing specialists have demonstrated how
irrelevant and potentially damaging it can be to insist on native
speaker norms when assessing the proficiency of English in the
Outer Circle( Lowenberg, 2000).
An argument that would have to be investigated for Expanding
Circle contexts too.
In-service Training- Session 4:
English as a Lingua Franca
ELF speech community should no longer be regarded as norm-dependent, but as norm-developing and norm-providing.
Speakers from different L1 backgrounds communicate through ELF and they co-construct ‘English’ in the process.
Degrees of cultural loadedness: As free as possible of prefabricated cultural baggage of inner circle countries and to create space for all cultures.
ELF aims at a formulation of minimal requirement for intelligibility and maximal flexibility for actual use. (Seidlhofer, 2002).
Progress Exam II
A transformative perspective
Transformation of language curriculum from native
based norms to ELF.
Open view of language models and standards,
challenging socially marked and restrictive models
Breaking the barriers imposed by inner-circle gate-
keepers in academia
Illustrative scales-Communicative Activities-
Overall Listening Comprehension
Understanding Interaction between Native Speakers
B2+ Can keep up with an animated conversation between native speakers.
B2 Can with some effort catch much of what is said around him/her, but may find it difficult to
participate effectively in discussion with several native speakers who do not modify their
language in any way.
B1 Can generally follow the main points of extended discussion around him/her, provided speech is
clearly articulated in standard dialect.
Illustrative scales-Overall Spoken Interaction
C2 Can understand any native speaker interlocutor, even
on abstract and complex topics of a specialist nature beyond
his/her own field, given an opportunity to adjust to a
non-standard accent or dialect.
B2 Can understand in detail what is said to him/her in the
standard spoken language even in a noisy environment.
Illustrative scales -Conversation
B2 Can sustain relationships with native speakers
without unintentionally amusing or irritating them
or requiring them to behave other than they would with a native
speaker.
Illustrative scales -Formal Discussion
C2 Can hold his/her own in formal discussion of complex issues,
putting an articulate and persuasive argument, at no
disadvantage to native speakers.
Structured overview of all CEFR scales
Native speaker: 16 times
Standard speech/dialect: 12 times
The term non-native: none
Conclusion
Language is an open, flexible, tool for communication
and identification with a community
The community of English language users worldwide is
growing so fast that it is transforming the way
communities of practice are seen. NNS teachers and
their perspective are contributing to this change by
challenging native speaker assumptions.
CEFR descriptor writers need to focus on this change
and include a perspective on ELF.
References
Cook, V. (1999). Going beyond the native speaker in language teaching. TESOL Quarterly, 33. 185-209.
Holliday, A. (1994). Appropriate Methodology and Social Context. Cambridge: Cambridge University Press.
House, J. (1999). Misunderstanding in intercultural communication: Interactions in English as a lingua franca and the myth of mutual intelligibility. In: Gnutzmann, C. (ed). Teaching and Learning English as a Global Language. Tubingen: Stauffenburg. 73-89.
Lowenberg, P.H. (2000). Non-native varieties and the sociopolitics of English proficiency assessment. In: Hall, J.K. and Eggington, E.G. (eds). The Sociopolitics of English Language Teaching. Clevedon: Multilingual Matters. 67-85.
Seidlhofer, B. (2002). ‘The shape of things to come? Some basic questions about English as a lingua franca’. In K. Knapp & C. Meierkord (Eds.), Lingua franca communication (pp. 269-302). Frankfurt/Main: Peter Lang.
Seidlhofer, B. (2005). ‘English as a lingua franca’. ELT Journal, 59/4: 339-441.
Widdowson, H.G.(1994). The ownership of English. TESOL Quarterly, 28. 377-389.
www.CEFtrain.net