CEFR Listening & Reading Assessment

Feedback on second pilot

Project AimsProject Aims

• Develop ideas for classroom assessment tasks for Listening & Reading at CEFR levels

• Stimulate use of authentic L & R materials • Develop and trial example tasks• Develop a set of (3-6?) most appropriate

templates so people can produce their own• Share materials between participating

schools to develop a bank of examples• Implement CEFR continuous assessment

Scenario - ObjectivesScenario - Objectives

Example at B1Example at B1

Example at B1 – Task 1Example at B1 – Task 1

Example at B1 – Task 2Example at B1 – Task 2

Example at B1 – TasksExample at B1 – Tasks

B1 Listening:Hampton Court Palace Teacher Version

• Hampton Court Palace Teacher Version Total Marks: 20 (15 with MCQ)The video is a trailer for the series “The Tudors.”

It is in three sections, with no breaks:

You need to use the video in two parts, stopping it when she meets the historian and his name appears on the screen.

*** Be careful that, if you introduce the topic, that you do not get into details about Henry VIII and his wives and give away the answers! In a London school, ask if anyone has been there (and make a note this of this on the score sheet)

Task 1: During 1st listening Listen to the introduction to a TV guided commentary on a historic place of interest “Hampton Court Palace,” near London. You will hear this introduction twice.Answer the 5 questions. Mark “T” if the statement is True “F” if it is False “NS” if the information is Not Stated in the commentary.

• Henry VIII, King of England, died 500 years ago. F• Henry VIII was one of the richest kings in Europe. NS• Henry bought Hampton Court from Cardinal Wolsey. F• Hampton Court was one of the most luxurious and modern palaces of its time.T• Henry had other palaces too, all connected by the river. T

Task 2:

Key

Task 3: After watching the video - Open Questions versionNow answer the following 5 questions with information from the commentary.

This is not a writing test.Grammar and spelling are not penalised.It is the information points that count. Answers in note form are acceptable.Alternative answers are possible. Suggestions are given.

• Why was this video about Hampton Court made? 2 marks for both(to celebrate the 500th anniversary) 1 mark for one(to advertise the series “The Tudors”)• What did Henry VIII use Hampton Court for? 2 marks for two

(for his court) 1 mark for one(to live in)(to entertain visitors)

• Why was the palace built on the river? (best transport / easier to travel) 2 marks

• What was the main topic that was discussed in Henry’s Council of State at Hampton Court?(ending his marriage/ getting a divorce) 2 marks

(his marriage/wives) 1 mark• Why was the lady (Catherine Howard) running through the palace shouting for Henry?

(she was afraid of being killed / she knew she would be killed) 2 marks for two(she was arrested and now had escaped) 1 mark for one(she wanted Henry to change his mind about killing her)

0r Multiple choice version:

Task 3: After watching the video. - MCQ versionNow answer the following 5 questions with information from the commentary.

• Why was this video about Hampton Court made?• because Hampton court has just been restored• because Hampton Court celebrated its 500th anniversary (b)• because the series “The Tudors” is being filmed at the moment• What did Henry VIII use Hampton Court for?

a) to live more safely than in Londonb) to have a private residence in the countryc) to house his court and entertain his visitors (c)

• Why was the palace built on the river?a) because Henry loved fishing.b) because rich people lived by the river.c) because travel by river was quicker. (c)

• What was the main topic they say was discussed in Henry’s Council of State at Hampton Court?a) his wars;

b) his marriage; (b) c) his finances.• Why was the lady (Catherine Howard) running through the palace shouting for Henry?

a) because she was afraid Henry was going to kill her. (a)b) because Henry was going to divorce her since she could not have a baby.c) because Henry had died and she was being sent to prison.

8

Assessment ideas

9

Feedback from first pilot• Suitability of content• Level• Workability

– Do the tasks work?– Instructions clear?– Timing?

• Presentation– Layout– Sound & video quality

• Scores 10

Second pilot• 12 schools received revised scenarios• Feedback from teachers by:

– Online survey– Paper questionnaire

• Individual scores collected

11

Editing existing scenarios• Suitability

– One scenario eliminated: cultural sensitivity• Workability

– One scenario eliminated: video no longer available– One completely revised: tasks re-ordered, re-

recorded• Level

– One scenario re-assigned from A2 to B1– Several minor edits

• Presentation– Changes to titles– Improved graphics (maps)– Editing terminology, fonts etc. for consistency

12

13

0 too easy for the stated level: suitable for the next level down

1 borderline easy: at lower end of level

2 clearly at this level

3 borderline hard: at upper end of this level

4 too hard for this level: suitable for next level up

LevelIs this text/task…

LevelAfter 1st pilot

14

Text average

Task average N

A1 Listening Visiting the Fashion Museum 2.25 2.25 4A1 Reading Gym application 1.00 1.00 1A1 Reading Postcard to Tom 2.00 2.50 2A2 Listening Visiting Dublin 2.57 2.71 8A2 Reading Choosing a hotel 3.25 2.67 4A2 Reading World news in brief 2.25 2.00 4B1 Listening Hampton Court 2.20 2.50 11B1 Listening Magic Breakfast 3.33 3.00 3B1 Listening One week job 2.00 2.25 4B1 Reading Cineworld Dublin      B1 Reading Rescued Russian children 2.50 2.50 2B2 Listening Advice for job seekers 2.67 2.83 6B2 Reading Online dating 2.20 2.00 5B2 Reading Women in the boardroom 2.00 2.00 1C1 Listening Do schools kill creativity? 2.33 3.00 6C1 Reading Bankers caused the crash 3.14 3.00 7

2.38 2.41 68

LevelAfter 2nd pilot

15

Text average

Task average N

A1 Listening Visiting the Fashion Museum 3.00 3.00 1A1 Reading Gym application 2.61 2.22 3A1 Reading Postcard to Tom 2.33 3.00 1A2 Listening Visiting Dublin 2.08 1.96 3B1 Reading Choosing a hotel 2.56 2.00 4A2 Reading World news in brief 2.00 2.00 1B1 Listening Hampton Court 2.25 2.25 4B1 Listening Magic Breakfast 2.83 2.08 4B1 Listening One week job 3.00 2.67 3B1 Reading Cineworld Dublin 2.00 2.00 1B1 Reading Rescued Russian children 2.67 2.17 3B2 Listening Advice for job seekers 3.04 3.25 4B2 Reading Online dating 2.00 2.00 1B2 Reading Women in the boardroom 2.25 2.17 2C1 Listening Do schools kill creativity? 3.00 2.42 2C1 Reading Bankers caused the crash 3.44 3.72 3

2.59 2.35 40

LevelComparison

16

Pilot 1 Pilot 2Text

averageTask

averageText

averageTask

average▼▲

A1 Listening Visiting the Fashion Museum 2.25 2.25 3.00 3.00 ▲A1 Reading Gym application 1.00 1.00 2.61 2.22 ▲A1 Reading Postcard to Tom 2.00 2.50 2.33 3.00 ▲A2 Listening Visiting Dublin 2.57 2.71 2.08 1.96 ▼B1 Reading Choosing a hotel 3.25 2.67 2.56 2.00 ▼A2 Reading World news in brief 2.25 2.00 2.00 2.00 ▼B1 Listening Hampton Court 2.20 2.50 2.25 2.25 ▼B1 Listening Magic Breakfast 3.33 3.00 2.83 2.08 ▼B1 Listening One week job 2.00 2.25 3.00 2.67 ▲B1 Reading Cineworld Dublin     2.00 2.00B1 Reading Rescued Russian children 2.50 2.50 2.67 2.17 ▼B2 Listening Advice for job seekers 2.67 2.83 3.04 3.25 ▲B2 Reading Online dating 2.20 2.00 2.00 2.00 ▼B2 Reading Women in the boardroom 2.00 2.00 2.25 2.17 ▲C1 Listening Do schools kill creativity? 2.33 3.00 3.00 2.42 ▲C1 Reading Bankers caused the crash 3.14 3.00 3.44 3.72 ▲

2.38 2.41 2.59 2.35 ▲

ScoresDetermining cut scores by aligning with teacher assessments: example

B2 Listening Advice for job seekers

B2 = 4

17

To do• Complete analysis of scores• Further edits in response to feedback

18