lessons from the fce/cae revision project - НБУold.nbu.bg/public/images/file/departments/foreign...
Post on 07-Feb-2018
235 Views
Preview:
TRANSCRIPT
-
Cambridge Whiffs of Change:
lessons from the FCE/CAE revision project
-
Lets benchmark!
What did they change?What did they not change?What considerations informed these decisions?Is there anything we can use to improve our B2 test?
-
150 years of assessment
Cambridge Assessment was established as theUniversity of Cambridge Local ExaminationsSyndicate (UCLES) in 1858 to administer localexaminations for students who were not members ofthe University of Cambridge
http://bp2.blogger.com/_0oOkTuc1hkU/R_DHAOcQzTI/AAAAAAAAABg/HotpowQGY8w/s400/cambridge.jpghttp://www.cambridgeassessment.org.uk/ca/digitalAssets/114053_ca-logo3.jpg
-
The CPE 1913 Examination
Oral: Dictation hourReading aloud and Conversation hour
Written:(a) Translation from English into French or German 2 hours(b) Translation from French or German into English, and questions on English Grammar 2 hours(c) English Essay 2 hours(d) English Literature 3 hours(e) English Phonetics 1 hours
-
CPE 1953 paper
Oral: Dictation, Reading aloud and Conversation
Written:(a) English Literature 3 hoursor Science Textsor English Life and Institutionsor Survey of Industry and Commerce
(b) Use of English 3 hoursor Translation from and into English
(c) English Language (composition and a passage of English with language questions) 3 hours
-
Certificate of Proficiency in English, English Life and Institutions paper, question 12:
It is said that the English love thesea.What effect has this had, in youropinion, on their way of life?
CPE candidates in an examination hall in Rome, June 1956
-
1946 LCE Total time: 5 hrs + the Oral
Oral: Dictation, Reading aloud, Conversation
Written: (a) Translation from and into Englishor Prescribed Books 2 hours (b) Composition and Language 2 hours
1975 FCETotal time: 5 hrs 55 m
Paper 1: CompositionPaper 2: Reading ComprehensionPaper 3: Use of EnglishPaper 4: Listening ComprehensionPaper 5: Interview
-
The charm of history and its enigmatic lesson consist in the fact that, from age toage, nothing changes and yet everything is completely different.
Aldous Huxley
-
The FCE and CAE Review Project20042008
CAMBRIDGE ESOL : RESEARCH NOTES : ISSUE 30 / NOVEMBER 2007
http://www.cambridgeesol.org/rs_notes/rs_nts30.pdf
Studies in Language Testing (SiLT) forthcoming volume
http://www.cambridgeesol.org/rs_notes/rs_nts30.pdf
-
FCE Paper 1: ReadingTime: 1 h 15 min 35 items
Part Task type Tests ability to No. Q
Multiple matching
A text preceded by a list of headings or summarising sentences for each paragraph
read for gist and main idea
6 or 7
Multiple choice A text followed by four-option questions
understand detail, opinion, deduce meaning
7 or 8
Gapped text A text with missing sentences or paragraphs, to be found from a jumbled list
understand text structure (coherence and cohesion)
6 or 7
Multiple matching
Several sections of text candidates match statements or questions about the text to the relevant sections
scan for specific information
13-15
-
FCE Paper 1: ReadingTime: 1 h 15 min 35 items
Part Task type Tests ability to No. Q
Multiple choice A text followed by four-option questions.
understand detail, opinion, deduce meaning, ..
7 or 8
Gapped text A text with missing sentences or paragraphs, to be found from a jumbled list.
understand text structure (coherence and cohesion)
6 or 7
Multiple matching Several sections of text candidates match statements or questions about the text to the relevant sections.
scan for specific information, ..
13-15
-
FCE Paper 1: ReadingTime: 1 h 30 items
Part Task type Tests ability to No. Q
Multiple choice A text followed by four-option questions.
understand detail, opinion, attitude, tone, purpose and main ideas, deduce meaning
8
Gapped text A text with missing sentences, to be found from a jumbled list.
understand text structure (coherence and cohesion)
7
Multiple matching
Several sections of text candidates match statements or questions about the text to the relevant sections.
scan for specific information, detail, opinion and attitude
15
-
FCE 2008Paper 1: Reading
Changes Rationalereduce time from 1 h 15 min (35
questions) to 1 h (30 questions)reduce overall text length only slightly
from 2,000-2,300 words to 2,000 wordsremove current Part 1 (matching
headings or summary sentences to paragraphs of a text)
broaden scope of multiple-choice tasks on Part 2 (now Part 1) text
use Part 3 gapped text task only in sentence form (not paragraphs)
time effectiveness: shorter format but broader scope
inclusion of essential reading skills: infer writers tone, attitude and purpose
simplified format: no alternative tasks, item number fixed by part
.
But why did the matching headings task go?
-
How to Make Tests More Reliable
Exclude items which do not discriminate well between weaker and stronger students.
[] A small number of non-discriminating items may be kept at the beginning of a test to give candidates confidence and reduce the stress they feel.
Hughes, Arthur. Testing for Language Teachers. CUP, 2003.
-
FCE Paper 2: WritingTime: 1 h 30 min
Part Task type Tests ability to No. words
Compulsory task
Read some input of up to 250 words and use the information to write either a formal or informal letter.
5 content points to be covered
request/give information, explain, describe, suggest, complain, apologise, express enthusiasm, gratitude, etc., in an appropriate register
120-180
Choice of four
A situationally-based task to write one of the following:
articlediscursive compositionletter reportstory
Final question based on set reading texts.
write for a specific purpose and audience, with accurate language control
120-180
-
FCE Paper 2: WritingTime: 1 h 30 min
Part Task type Tests ability to No. words
Compulsory task
Read some input of up to 250 words and use the information to write either a formal or informal letter, or
5 content points to be covered
request/give information, explain, describe, suggest, complain, apologise, express enthusiasm etc. in an appropriate register
120-180
Choice of four
A situationally-based task to write one of the following:
articlediscursive compositionletter reportstory..
Final question based on set reading texts.
write for a specific purpose and audience, in a specified genre
120-180
-
FCE Paper 2: WritingTime: 1 h 20 min
Part Task type Tests ability to No. words
Compulsory task
Read some input of up to 160 words and use the information to write either a formal or informal letter, or an e-mail.
4 content points to be covered
request/give information, explain, describe, suggest, complain, apologize, express enthusiasm etc. in an appropriate register
120-150
Choice of four
A situationally-based task to write one of the following:
articleessayletter reportstoryreview
Final question based on set reading texts.
write for a specific purpose and audience, in a specified genre
120-180
-
FCE 2008Paper 2: Writing
Changes Rationalereduce time from 1 h 30 min to 1 h 20
minreduce input for Part 1reduce output for Part 1 from 180 to 150 broaden scope by
including production of emailsinclude review task in Part 2
reduce set texts from five to two, with questions related to specific book
reinstate the essay
writing more briefly is a relevant real life skill writing email messages is a relevant life skill
review is appropriate to the candidature in terms of both experience and lang. skills
On VRIP analysis,smaller input and output was found not to
affect adversely the validity and reliability of the exam
smaller input and output was found to affect positively the impact and practicality of the exam
But why didnt the set books go?
-
Backwash: advantages to be gained from studying set texts at FCE and CAE level
To provide a source of pleasure and stimulationTo encourage a lifelong interest in readingTo widen cultural and literary horizonsTo promote contact with other media: film, theatre, etc.To engage the reader on a subconscious level in terms of structure, vocab, collocation, which may influence the readers own writing skills
-
FCE Paper 3: Use of EnglishTime: 1 h 15 min 65 items
Part Task type Tests knowledge of No. Q
Multiple choice cloze
Choose 1 word from a set of 4 to complete a gap in a text.
collocations, phrasal verbs, idioms, phrases
15
Open cloze Think of one word to complete a gap in a text.
grammatical items 15
Word formation Form a new word from a given stem word to complete a text.
prefixes and suffixes 10
Key word transformations
Complete a paraphrase of a given sentence using 2 to 5 words incl. a given key word.
grammatical and lexical items
10
Error correction Read a text containing errors identify one extra word in some lines, mark others as correct.
proofread and edit a text for grammatical correctness
15
-
FCE Paper 3: Use of EnglishTime: 1 h 15 min 65 items
Part Task type Tests knowledge of No. Q
Multiple choice cloze
Choose 1 word from a set of 4 to complete a gap in a text.
collocations, phrasal verbs, idioms, phrases
15
Open cloze Think of one word to complete a gap in a text.
mostly grammatical items 15
Word formation Form a new word from a given stem word to complete a text.
prefixes and suffixes 10
Key word transformations
Complete a paraphrase of a given sentence using 2 to 5 words incl. a given key word.
grammatical and lexical items
10
-
FCE Paper 3: Use of EnglishTime: 45 min. 42 items
Part Task type Tests knowledge of No. Q
Multiple choice cloze
Choose 1 word from a set of 4 to complete a gap in a text.
collocations, phrasal verbs, idioms, phrases
12
Open cloze Think of one word to complete a gap in a text.
mostly grammatical items 12
Word formation Form a new word from a given stem word to complete a text.
prefixes and suffixes 10
Key word transformations
Complete a paraphrase of a given sentence using 2 to 5 words incl. a given key word.
grammatical and lexical items
8
-
FCE 2008Paper 3: Use of English
Changes Rationalereduced from 1 h 15 min
(65 questions) to 45 minutes (42 questions) - about 40% less
remove Part 4 (error correction)
time effectivenessthe grammar that the error
correction task tests is tested elsewhere in the paper
the skill of proofreading and editing can be tested through the writing paper
But why did they play down Use of English importance?
-
FCE Paper 4: ListeningTime: 40 min
Part Task type Tests ability to understand
No Q
Multiple choice
Listen to 8 unrelated dialogues of approx. 30 sec. each answer a three-option question on each.
function, location, roles and relationships, mood, attitude, intention, feeling or opinion
8
Sentence or note completion
Listen to a monologue or dialogue of about 3 minutes complete sentences or notes with info heard.
main points, opinion, detail or specific information, deduce meaning
10
Multiple matching
Listen to 5 topic-related monologues of 30 seconds each. Match each speaker to the correct option from a list of 6.
specific information, understand attitude and/or events from the text
5
Multiple choice orTrue/False
Listen to a 3-minute monologue or dialogue and answer three-option questions about it, or mark statements as true/false.
main points, opinion, detail or specific information, deduce meaning
7
-
FCE Paper 4: ListeningTime: 40 min
Part Task type Tests ability to understand
No Q
Multiple choice
Listen to 8 unrelated dialogues of approx. 30 sec. each answer a three-option question on each.
function, location, roles and relationships, mood, attitude, intention, feeling or opinion
8
Sentence or notecompletion
Listen to a monologue or dialogue of about 3 minutes complete sentences or notes with info heard.
main points, opinion, detail or specific information, deduce meaning
10
Multiple matching
Listen to 5 topic-related monologues of 30 seconds each match each speaker to the correct option from a list of 6.
specific information, understand attitude and/or events from the text
5
Multiple choice orTrue/False
Listen to a 3-minute monologue or dialogue answer three-option questions about it, or mark statements as true/false.
main points, opinion, detail or specific information, deduce meaning
7
-
FCE Paper 4: ListeningTime: 40 min
Part Task type Tests ability to understand
No Q
Multiple choice
Listen to 8 unrelated dialogues of approx. 30 sec. each answer a three-option question on each.
function, location, roles and relationships, mood, attitude, intention, feeling or opinion
8
Sentence completion
Listen to a monologue or dialogue of about 3 minutes complete sentences with info heard.
main points, opinion, detail or specific information, deduce meaning
10
Multiple matching
Listen to 5 topic-related monologues of 30 seconds each match each speaker to the correct option from a list of 6.
specific information, understand attitude and/or events from the text
5
Multiple choice
Listen to a 3-minute monologue or dialogue answer three-option questions about it.
main points, opinion, detail or specific information, deduce meaning
7
-
FCE 2008Paper 4: ListeningChanges Rationale
only sentence completion in Part 2 (note-taking removed as an option)
only multiple-choice in Part 4 (true-false removed as an option)
sentence completion more suitable to the level than note-taking
standardised format will improve reliability by ensuring comparability between test versions
standardised format has a positive impact on test-takers and teachers
Ockhams razor: Entities are not to be multiplied without necessity.
Make everything as simple as possible, but not simpler.Albert Einstein
-
FCE Paper 5: SpeakingTime: ~14 min
Part Task type Tests ability to Length
Interview The interlocutor asks each candidate questions about their own life.
use language for social purposes
3 min.
Individual long turn (1 min) and response (20 sec)
The first candidate is given two pictures to talk about for 1 minute. The second candidate makes a short response, then the roles are reversed.
describe, compare and contrast, speculate, express opinion
1 min. + 20 sec. each
Two-way collaborative task
Candidates are given spoken instructions, along with visual and written prompts, and work on a decision-making task.
invite/give opinions, agree/disagree, ask for/make suggestions, evaluate options, take turns, negotiate a decision
3 min.
Three-way discussion
The interlocutor starts a discussion on the topic in Part 3
give & justify opinions, agree & disagree, take turns and develop a discussion
4 min.
-
FCE 2008Paper 5: Speaking
Changes Rationaleadd written replication of
task instructionswill help candidates recall
the taskswill improve objectivity and
impact
-
CAE Paper 1: ReadingTime: 1 h 15 min 45 items
Part Task type Tests ability to No. Q
Multiple matching
A text with several sections candidates match statements or questions about the text to the relevant sections.
scan for specific information
12-18
Gapped text A text with missing paragraphs which have to be found from among a jumbled list.
understand text structure (coherence and cohesion)
6-7
Multiple choice A longer text followed by four-option questions.
find detail, recogniseinference, and interpret the text
5-7
Multiple matching
A text with several sections candidates match statements or questions about the text to the relevant sections.
scan for specific information
12-22
-
CAE Paper 1: ReadingTime: 1 h 15 min 34 items
Part Task type Tests ability to No. Q
Multiple choice 3 short themed texts, each followed by 2 four-option questions (as in CPE)
understand detail, opinion, tone, purpose, main idea, attitude, text organization (exemplification, comparison, reference)
6
Gapped text A text with missing paragraphs which have to be found from among a jumbled list.
understand text structure (coherence and cohesion)
6
Multiple choice A longer text followed by four-option questions.
find detail, recogniseinference, and interpret the text
7
Multiple matching
A text with several sections candidates match statements or questions about the text to the relevant sections.
scan for specific information
15
-
CAE 2008Paper 1: Reading
Changes Rationalereduce number of questions from
c. 45 (1 h 15 min) to 34 (1 h 15 min)minimal change in word length to
c. 3,000 wordsremove current Part 1 (multiple
matching)introduce themed texts (as in CPE
Paper 1)fix the no. of questions per part
avoid overlap in testing focus (former Part 1 = Part 4)
widen range of genres and registers: positive backwash effect
exam family resemblance: ensure a more straightforward progression from FCE through to CPE
-
CAE Paper 2: WritingTime: 2 h
Part Task type Tests ability to No. words
Compulsory task
Read some input of up to 400 words and use the information to write one or two of the following: letter, report, proposal, article, info sheet, brochure, memo, leaflet, e-mail.
evaluate, express opinions, hypothesize, persuade; in an appropriate register and style
250
Choice of four
A situationally-based task to write one of the following: letter, report, proposal, article, info sheet, review, contribution to a longer piece, brochure, memo, leaflet, discursive composition, competition entry.
Final question based on set reading texts.
write for a specific purpose and audience in a specified genre; varying focuses incl. comparing, giving advice, justifying, persuading
250
-
CAE Paper 2: WritingTime: 1 hour 30 min
Part Task type Tests ability to No. words
Compulsory task
Read some input of up to 150 words and use the information to write one of the following: letter, report, proposal, article.
evaluate, express opinions, hypothesize, persuade; in an appropriate register and style
180-220
Choice of four
A situationally-based task to write one of the following: letter, report, proposal, article, info sheet, review, contribution to a longer piece, essay.
Final question based on set reading texts.
write for a specific purpose and audience in a specified genre; varying focuses incl. comparing, giving advice, justifying, persuading
220-260
-
CAE Paper 3: English in UseTime: 1 h 30 min 74 items
Part Task type Tests knowledge of No. Q
Multiple choice cloze
Choose 1 word from a set of 4 to complete a gap in a text.
collocations, phrasal verbs, idioms, phrases
15
Open Cloze Think of one word to complete a gap in a text.
grammatical items 15
Word formation Form a new word from a given stem word to complete a text.
prefixes and suffixes 13
Error correction Identify extra word OR correct spelling & punctuation.
proofreading and editing 15
Register transfer Use a text on one register to complete a text in another.
register 10
Discourse cloze Choose a sentence or phrase to complete a text from a list of 9.
text organisation 6
-
CAE Paper 3: Use of EnglishTime: 1 h 50 items
Part Task type Tests knowledge of No. Q
Multiple choice cloze
Choose 1 word from a set of 4 to complete a gap in a text.
collocations, phrasal verbs, idioms, phrases
12
Open Cloze Think of one word to complete a gap in a text.
grammatical items 15
Word formation Form a new word from a given stem word to complete a text.
prefixes and suffixes 10
Gapped sentences
Three discrete sentences with a gap each write the gapped word that is common to all three sentences.
polysemy 5
Key word transformations
Complete a paraphrase of a given sentence using 3 to 6 words incl. a given key word.
grammatical and lexical items
8
-
CAE Paper 4: ListeningTime: 40 min
Part Task type Tests ability to understand
No Q
Sentence ornote completion
Listen to a monologue of approx. 2 min. twice complete sentences or take notes.
specific information 8-10
Sentence ornote completion
Listen to a monologue of approx. 2 min. ONCE complete sentences or take notes.
specific information 8-10
Multiple matching
Listen to 5 themed monologues of 30 sec. each do two multiple matching tasks , each requiring selection of 8.
gist, attitude, main points; interpret context; deal with a variety of accents and styles of delivery
10
Multiple choiceor sentence completion
Listen to a conversation b/n 2 or 3 speakers of approx. 4 min. complete sentences or answer four-option questions about it.
gist, specific information and attitude
6-12
-
CAE Paper 4: ListeningTime: 40 min
Part Task type Tests ability to understand
No Q
Multiple choice: three exchanges
Listen to 3 short dialogues answer 2 three-option questions on each.
feeling, attitude, opinion, purpose, function, agreement, course of action, gist, detail
6
Sentence completion
Listen to a monologue of approx. 3 min. twice complete sentences or take notes.
specific information 8
Multiple matching
Listen to 5 themed monologues of 30 sec. each do two multiple matching tasks , each requiring selection of 8.
gist, attitude, main points; interpret context; deal with a variety of accents and styles of delivery
10
Multiple choice Listen to a conversation b/n 2 or more speakers of approx. 4 min. answer four-option questions.
gist, specific information and attitude
6
-
Is there anything we can change in the NBU B2 test?
One must change one's tactics every ten years if one wishes to maintain one's superiority.
- Napoleon Bonaparte
-
Do we really want toattach greater importance to Use of English (30 p.) than Listening (15 p.) or Writing (15 p.)?test the skill of deducing meaning from context 7 times over (out of 18 shots) rather than broaden the scope of skills tested?test reading for gist (match headings), for one half of the reading test (12 points out of 24)?avoid testing coherence and cohesion at the receptive level while highlighting it at the productive level?compromise reliability by having 9 True/False questions in the Listening (out of 15)?have one single text in the C1Listening rather than broaden scope (dialogue/monologue, etc.) and give test-takers a fresh start with each of several (cf. CAE -10)
-
Wouldnt perhaps the NBUtest be better served by
......
Change does not necessarily assure progress, but progress implacably requires change.
-
NBU seminar
October 29th, 2008Presenter: Danya Doganova
Cambridge Whiffs of Change:Lets benchmark!The FCE and CAE Review ProjectFCE Paper 1: Reading Time: 1 h 15 min 35 items FCE Paper 1: Reading Time: 1 h 15 min 35 items FCE Paper 1: Reading Time: 1 h 30 items FCE 2008Paper 1: ReadingFCE Paper 2: Writing Time: 1 h 30 min FCE Paper 2: Writing Time: 1 h 30 min FCE Paper 2: Writing Time: 1 h 20 min FCE 2008Paper 2: WritingBackwash: advantages to be gained from studying set texts at FCE and CAE levelFCE Paper 3: Use of English Time: 1 h 15 min 65 items FCE Paper 3: Use of English Time: 1 h 15 min 65 items FCE Paper 3: Use of English Time: 45 min. 42 items FCE 2008Paper 3: Use of EnglishFCE Paper 4: Listening Time: 40 min FCE Paper 4: Listening Time: 40 min FCE Paper 4: Listening Time: 40 minFCE 2008Paper 4: ListeningFCE Paper 5: Speaking Time: ~14 minFCE 2008Paper 5: SpeakingCAE Paper 1: Reading Time: 1 h 15 min 45 items CAE Paper 1: Reading Time: 1 h 15 min 34 items CAE 2008Paper 1: ReadingCAE Paper 2: Writing Time: 2 hCAE Paper 2: Writing Time: 1 hour 30 minCAE Paper 3: English in Use Time: 1 h 30 min 74 items CAE Paper 3: Use of English Time: 1 h 50 items CAE Paper 4: Listening Time: 40 minCAE Paper 4: Listening Time: 40 minIs there anything we can change in the NBU B2 test?Do we really want toWouldnt perhaps the NBU test be better served by NBU seminar
top related