m1u6 lecture.ppt
TRANSCRIPT
-
8/10/2019 M1U6 Lecture.ppt
1/42
ASSESSING SPEAKING PURPOSES AND
TECHNIQUES
Prepared by Elena Onoprienko, Yulia Polshina, TatianaShkuratova
Based on material by Fumiyo Nakatsuhara
-
8/10/2019 M1U6 Lecture.ppt
2/42
Outline
Key questions
Nature of speaking
Speaking as a skill Test purposes and types of test
Speaking test tasks
Scoring Washback effect
-
8/10/2019 M1U6 Lecture.ppt
3/42
KEY QUESTIONS
-
8/10/2019 M1U6 Lecture.ppt
4/42
Key questions
Construct Purpose
Task types Scoring criteria
How(score)?
Why assess
speaking?
How(test)?
What is
speaking?
-
8/10/2019 M1U6 Lecture.ppt
5/42
THE NATURE OF SPEAKING
-
8/10/2019 M1U6 Lecture.ppt
6/42
Nature of speaking:
spoken language;
speaking as interaction;
speaking as a social activity; speaking as a situation-based activity.
-
8/10/2019 M1U6 Lecture.ppt
7/42
What is speaking?
A part of the shared social activity of talking
(Luoma, 2004: 29).
In comparison with writing, speakingisMore: transient
dynamic interpersonal
content dependent.
Less:
planned
complex formal
lexically dense.
-
8/10/2019 M1U6 Lecture.ppt
8/42
Speaking vs Writing
The main differences are in two sets ofconditions - processingand reciprocity:
Processingis connected with time - speakingis going on under greater pressure of time.
Solution to this problem in spoken languagereciprocity. Speakers take turns and create a
text together.(Bygate,1987)
-
8/10/2019 M1U6 Lecture.ppt
9/42
Spoken language
Pronunciation
Spoken grammar
Lexis
-
8/10/2019 M1U6 Lecture.ppt
10/42
Pronunciation
Speech is judged on the basis of pronunciation.
What is standard? Native speaker vs non-native speaker.
Communicative effectiveness, which is based on
comprehensibility and probably guided by native speakerstandards but defined in terms of realistic learnerachievement, is a better standard for learner pronunciation.(Luoma, 2004).
What to include in assessment of pronunciation?
Pronunciationindividual sounds, pitch, volume, speed,pausing, stress and intonation.
-
8/10/2019 M1U6 Lecture.ppt
11/42
Spoken grammar:
grammar is easy to judge because it is easy todetect in speech and writing;
speakers do not usually speak in sentences;
speech consists of idea units connected with and,or, but, or that;
planned vs unplanned speechcomplexstructures vs short idea units;
the internal structure of idea units - topicalisationand tailscreate an impression of naturalness.
-
8/10/2019 M1U6 Lecture.ppt
12/42
Features of spoken lexis:
simple and ordinary words are common innormal spoken discourse and mark a highlyadvanced level of speaking skills (Luoma, 2004);
generic words (important for the naturalness oftalk);
vague words;
fixed conventional phrases; small words (the more the better perceived
fluency).
-
8/10/2019 M1U6 Lecture.ppt
13/42
Slips and errors
Normal speech contains a fair number ofslips and errors such as mispronouncedwords, mixed sounds, and wrong words
due to inattention (Luoma, 2004).
-
8/10/2019 M1U6 Lecture.ppt
14/42
SPEAKING AS A SKILL
-
8/10/2019 M1U6 Lecture.ppt
15/42
Speaking as a skill
What is skillful speech?
task fulfillment/content;
fluency;accuracy;
vocabulary and grammar range;
interaction.
-
8/10/2019 M1U6 Lecture.ppt
16/42
Speaking as meaningful interaction
Speaking is both personal and a part of theshared social activity of talking.
The openness of meanings is not only aconvenience in speech; it is also an effectivestrategy for speakers. (Luoma, 2004)
Chatting vs information-related talk.
The role of speaking situations. Roles, role relationships and politeness.
-
8/10/2019 M1U6 Lecture.ppt
17/42
Why assess speaking?
No single answer:
different groups of language learners have different needs, suchas:
international travellers: language for travel, leisure;
migrants: survival skills, access to employment;
students: exams, academic communication, socialinteraction;
professionals: workplace communication, presentations.
different users have different purposes when they seekinformation from tests;
but most users of language do need to speak.
-
8/10/2019 M1U6 Lecture.ppt
18/42
TEST PURPOSES AND TYPES OF TEST
-
8/10/2019 M1U6 Lecture.ppt
19/42
Test purposes and types of test
Test purposes:
proficiency tests
achievement tests placement tests
diagnostic tests.
-
8/10/2019 M1U6 Lecture.ppt
20/42
What do we need to decide beforegiving a speaking test?
what aspects of languagewe want to assess;
how to elicit ratable language samplesfrom test-takers suitable for the aspects of language.
We need to decide:
rating criteria[marking categories, levels,descriptors] [holistic scales vs. analytical scales];
elicitation techniques / test format(types ofquestions, task types).
-
8/10/2019 M1U6 Lecture.ppt
21/42
Performance testing
Performance testing in second languageproficiency assessment is traditionally usedto describe the approach in which a
candidate produces a sample of spoken orwritten language that is observed andevaluated by an agreed judging process.
(McNamara, 1996)
-
8/10/2019 M1U6 Lecture.ppt
22/42
What is performance testing?
sample of written or spokenlanguage;
simulates behaviour in the real world- not
like paper-and-pencil objective tests; observed and evaluated by an agreed
judging process.
-
8/10/2019 M1U6 Lecture.ppt
23/42
Speaking tasks
A communicative task is a piece of classroom work
which involves learners in comprehending,
manipulating, producing or interacting in the target
language while their attention is principally focusedon meaning rather than form (Nunan 1993:59).
Speaking tasks can be seen as activities that involve
speakers in using language for the purpose of
achieving a particular goal or objective in a particular
speaking situation (Bachman and Palmer 2010).
-
8/10/2019 M1U6 Lecture.ppt
24/42
Types of information-related talk
Factually-oriented talk:
description
narration instruction
comparison.
Evaluative talk:
explanation
justification prediction
decision.
(Bygate,1987)
-
8/10/2019 M1U6 Lecture.ppt
25/42
-
8/10/2019 M1U6 Lecture.ppt
26/42
Features of a speaking task:
input, or material used in the task;
roles of the participants;
settings, or classroom arrangements for paired or group
work; actions, or what is to happen in the task;
monitoring, or who is to select input, choose role or setting,alter actions;
outcomes as the goal of the task;
feedback given as evaluation to participants.
Candlin (1987) cited by Fulcher (2003)
-
8/10/2019 M1U6 Lecture.ppt
27/42
Speaking test task formats
Individual
Paired
Group
Open-ended tasks
Structured tasks
-
8/10/2019 M1U6 Lecture.ppt
28/42
Advantages and disadvantages ofan interview
+ testers control over interaction
+ opportunity for an examinee to show the
range of their speaking skills- it is costly in terms of testers time
- interviewers power over an examinee
-
8/10/2019 M1U6 Lecture.ppt
29/42
Advantages and disadvantages ofpaired formats
+Capable of eliciting more symmetrical contributionto theinteraction from test-takers
+Capable of eliciting much richer and more varied languagefunctions
+ Positive reactionfrom test-takers (less anxious), a sign ofpositive washbackeffect
+ Practical: time-efficient, cost-effective, less burden and lesstraining for the examiners
-The amount of responsibilityon examinees who are nottrained in interview techniques
-
8/10/2019 M1U6 Lecture.ppt
30/42
Advantages and disadvantages ofgroup formats
+ Well-received by learners
+ Support learning
- Difficult to administer and manage (sizeof the groups and mixture of learnersabilities)
- Difficult to monitor the progress of the
testing
-
8/10/2019 M1U6 Lecture.ppt
31/42
SPEAKING TEST TASKS
-
8/10/2019 M1U6 Lecture.ppt
32/42
Speaking test tasks:
oral presentation (verbal essay, prepared monologue);
information transfer (description of picture sequence,questions on a single picture, alternative visual stimuli);
interaction tasks (information gap: studentstudent,studentexaminer, open role play, guided role play);
interview (free, structured);
discussion (student-student, student-examiner).
(OSullivan, 2008: 10-11)
-
8/10/2019 M1U6 Lecture.ppt
33/42
Framework for designing test tasks
Operations (activities/skills) - informational routines (e.g.telling a story) and improvisational skills (negotiation ofmeaning and management of interaction)
Conditionsunder which the tasks are performed (e.g. timeconstraints, the number of people involved and familiaritywith each other)
Quality of output, the expectedlevelof performance interms of various relevant criteria, e.g. accuracy, fluency or
intelligibility.(Weir, 1993: 30)
-
8/10/2019 M1U6 Lecture.ppt
34/42
-
8/10/2019 M1U6 Lecture.ppt
35/42
Developing criteria for assessingspeaking
The importance of double marking for reducing unreliabilityis undeniable.
These criteria need to reflect the features of spokenlanguage interaction the test task is designed to generate.
The criteria used would depend on the nature of the skillsbeing tested and the level of detail desired by the end users.The crucial question would be what the tester wants to findout about a students performance on appropriate spoken
interaction tasks.(Weir, 1993, p.30)
-
8/10/2019 M1U6 Lecture.ppt
36/42
Rating criteria
Phonological control; Grammatical accuracy; Vocabulary range; Fluency(Council of Europe 2001)
Test format:interview formatwith the following structure:
1.Openings(1 minute).
2.Conversation on familiar topics (3 minutes)The interviewer asks thecandidate to talk about him/herself.
3.Picture Description (2 minutes)The interviewer asks the candidate todescribe a photo.
4.Conversation on topics from the given picture(5 minutes) The interviewer
asks the candidate questions linked to the picture (from general to extendedquestions).
5.Closings (1 minute). (Nakatsuhara, 2012)
-
8/10/2019 M1U6 Lecture.ppt
37/42
Scoring
Holistic scale
e.g. Trinity College
Bands A, B, C, D
Analytic scale
e.g. IELTS
Fluency and coherence
Lexical resources
Grammatical range andaccuracy
Pronunciation
-
8/10/2019 M1U6 Lecture.ppt
38/42
Holistic rating scales
Positive features:
practicality: fast;
rating holistically may be more naturalistic.
Disadvantages:
no useful diagnostic information: single score;
not always easy to interpret: raters not requiredto use same criteria to arrive at score.
-
8/10/2019 M1U6 Lecture.ppt
39/42
Analytic rating scales
Positive features:
can provide diagnostic information if scores reportedseparately;
potentially clear, explicit and detailed; usually more reliable (multiple scores);
useful in training raters to focus on our construct;
potentially useful in guiding learners.
Disadvantages: time-consuming;
may overburden raters. (Green, 2012)
-
8/10/2019 M1U6 Lecture.ppt
40/42
The role of an interviewer
Interrater/ intrarater reliability
The solutiontraining raters:
understanding criteria for assessment;
agreement with other raters;
consistency of performance.
-
8/10/2019 M1U6 Lecture.ppt
41/42
WASHBACK EFFECT
-
8/10/2019 M1U6 Lecture.ppt
42/42
Washback Effect:
The effect of testing on teaching and learning
Positive / negative washback:
positivetest stimulates classroomteaching of important skills;
negativenarrow focus on teaching just forthe test.