trincheri, miranda / consorcio else brown, ana ... · (brasil, 2015, p.9) ... contest described in...
TRANSCRIPT
TRINCHERI, Miranda / Consorcio ELSE BROWN, Ana / Laboratorio de Idiomas, UBA
IES Lenguas Vivas
14th EALTA Conference
Language testing and assessment Its role in mobility and social integration 1-3 june 2017, Sèvres, France
CELU certification: Introduction, development
CELU / CELPE-Bras: regional integration in SA
Construct, theoretical view
Test design: integrated tasks
Task assessment
Analysis: tasks and its parts
Conclusions: impact
Argentina´s official certification of SpanishL2.
Officially acknowledged by the Ministry of Education and by the Ministry of Foreign Affairs of Argentina.
Started in 2004 and been growing since.
Administered by the ELSE Consortium.
More than 50 exam centres
Bakhtin´s perspective on social use of language (Bajtin, 1982).
Critical language testing (Shohamy,1998)
Performance test (Alderson, 1995; Mc Namara, 2000; Schlatter et.al., 2005)
Impact (Bachman, 1996; Watanabe, 2004).
Proficiency (Dell'Isola et.al., 2003)
“Speakers can participate in
different activities within
their resources with or
without help from others”.
“Different levels of
achievement show different
levels of proficiency in
language use"
An exam of language
use
Dell’Isola et.al., 2003
Based on a concept of language as a combined action of participants with a social aim and also, considering language and culture as inseparable, the idea of proficiency that underlies the exam consists of the adequate use of language in order to perform acts in the real world. Celpe-Bras considers context and aim of the participants in the interaction.
(Brasil, 2015, p.9)
Reciprocal agreement between Argentina and Brasil governments to
acknowledge both exams. (05/08/2005)
8
Format
Part Skills Timing Activities
Written
Reading
Listening
Writing
3 hours
Listening and writing with a particular
purpose to a specific audience
No explicit questions about the language
or its grammar are asked
Oral
Reading
Listening
Interacting
15 to 20 minutes Reading short texts, commenting on them
while interacting with the examiner
Intermediate
Language users at the
Intermediate level can speak and
write in Spanish quite fluently
and naturally, in simple family,
working and social
environments.
They may lack accuracy when
facing unknown situations or
when having to express
subtleties of meaning.
They manage to communicate
in working as well as academic
contexts.
Advanced
Language users at the Advanced
level can speak and write in
Spanish with ease and
spontaneity in a wide range of
social situations.
They can use the language
appropriately in working as well
as academic contexts.
CELU levels
Full descriptions: http://www.celu.edu.ar/en/content/levels
Genre organizes language experience.
Skills, as in real life, can be assessed from an integrated perspective.
Tasks expect adequate answers according to a genre, a role and a purpose. Action-oriented and contextualized tasks.
You are one of the artists that participated in the contest described in the article and you didn´t win. Write an e-mail to the judges to express your disagreement with the results.
Usted es uno de los artistas que se presentó al concurso que se menciona en la siguiente noticia y no ganó. Escríbale un correo electrónico al presidente del jurado para expresar su disconformidad por el resultado.
You were part of the expedition team mentioned in the following text. Tell on your travel blog how the expedition to the garbage island was and comment on your experience.
Usted formó parte del equipo de exploradores mencionado en el siguiente
texto. Cuente en su blog de viajes cómo fue la expedición a la isla de la basura y comente su experiencia.
Each task is assessed by two independent raters.
Raters attend standardisation sessions for every exam.
Holistic rating scales with five different performance levels.
Expected answer
A sample of answers is collected and their level is agreed among raters.
Basic (2) Advanced (5) The text doesn´t fulfill genre or purpose. Relationship between his/her role and the addressee is not adequate or is different from the one required. Recontextualizes inadequately or wrongly input information. Or the texts lacks cohesion and coherence. Linguistic resources are very limited.
Fulfills the purpose of the task. Builds an adequate relationship between his/her role and the addressee. Finds a position within the genre and produces a personal text. Retrieves information from the input. Rewords and organizes input information, following required genre characteristics. The text is coherent and cohesive. Lexicon and structures are varied and adequate. Could be occasional inadequacies which not affect reading fluency.
In which ways the candidate plays an active role in the language that it´s being tested?
We define active role as social actions that are
embedded in the linguistic choice the candidate
make. For example, the means chosen to fulfill the
purposes of the text.
Methodology
Analysis of tasks (2014/2015)
-Required genres
-Roles
-Purposes
CELU 114 CELU 214 CELU 115 CELU 215
June 2014
November 2014
June 2105
November 2015
4 exams, 16 written tasks
INPUT OUTPUT
ROLE ADDRESSEE
PURPOSE DOMAIN
ACT INPUT OUTPUT ROLE ADDRESSEE PURPOSE DOMAIN
1 Interview Article School
principal Specialized magazine
Spread PUBLIC
2 News E-mail (f) Artist Judges Express
disagreement PRIVATE
3 Article E-mail (inf) Friend Friend Comment and advice
PRIVATE
4 News Letter to the
editors Motorist Newspaper Give opinion PUBLIC
The exam offers the candidate a range of social processes to interact with
The role and the purpose assigned in the task allow the candidate to interact with different domains: public and private, personal and professional
All the tasks are designed to promote linguistic and social activity interaction as a mean of assessment
ELSE Consortium: 3 universities (2004), 38 universities (2017)
Training of examiners all over the world, academic network focused on SL2 assessment
Research: periodic conferences, studies and publications
CELU Archive: it gathers all the exams and it´s used by researchers. Currently, more than 18,000 written texts and 2500 interviews being coded.
Consortium activities: language courses, class materials, workshops, etc.
Critical language testing assumes that the act of testing is not neutral. Rather, it is both a product and an agent of cultural, social, political, educational and ideological agendas that shape the lives of individual participants, teachers and learners (Shohamy, 1998, 336).
Alderson, J. C., Clapham, C., & Wall, D. (1995). Language test construction and evaluation. Ernst Klett Sprachen.
Bajtin, M. M. (1982). Estetica de la creacion verbal. Mexico: Siglo veintiuno editores.
Bachman, L. y Palmer, A. (1996). Language testing in practice: designing and developing useful language tests,
Oxford, Oxford University Press.
Brasil (2015). Certificado de Proficiência em Língua Portuguesa para Estrangeiros: Manual do Aplicador. Secretaria de
Educação Superior (SESu). Brasília: MEC. Available at:
http://www.ufrgs.br/acervocelpebras/arquivos/manuais/manual-do-aplicador-2015
CELPE Bras Web site http://celpebras.inep.gov.br/inscricao/
CELU Web site http://www.celu.edu.ar/
Dell'Isola, R. L., Scaramucci, M. V., Schlatter, M., & Júdice, N. (2003). A avaliação de proficiência em português
língua estrangeira: o exame CELPE-Bras. Revista Brasileira de Linguística Aplicada, 3(1), 153-164.
Mc Namara, T (2000) Language testing, Oxford, Oxfod University Press.
Schlatter, M., Almeida, A. D. N., Fortes, M. S., & Schoffen, J. R. (2005). Avaliação de desempenho e os conceitos de
validade, confiabilidade e efeito retroativo. A redação no contexto do vestibular, 11-35.
Shohamy, E. (1998). Critical language testing and beyond. Studies in educational evaluation, 24(4), 331-345.
Watanabe, Y. (2004). “Methodology in washback studies” en Cheng, L; Watanabe,Y; Curtis, A (eds), Washback in
language testing, research contexts and methods, London, Lawrence Erlbaum.
Ana Brown / Laboratorio de Idiomas, UBA
& IES Lenguas Vivas
Miranda Trincheri / Universidad de Buenos Aires & Consorcio ELSE