CEM Conference8 June 2011

Using CEM data:Convincing colleagues, pupils & parents

Bands, percentiles, standardised scores…

0

50 60 70 80 90 100 110 120 130 140 150

Standardised scores

DC B

A

105 20 301 40 90 958070 996050Percentiles:

Band Profile Graph: all MidYIS cohort

Current Year 10

Overall: Percent in each BandYear 9/S2 MidYIS Test 2009/2010

15

15

79

0

10

20

30

40

50

60

70

80

90

100

D C B A

Per

cen

t

D C B A0

10

20

30

40

50

60

70

80

90

100

03

16

81

Per

cen

tBand Profile Graph: all MidYIS

cohort

Current Year 11

The Computer Adaptive Baseline Test (CABT)

In 2009 / 2010 approx 200,000+ students sat the test

Vocabulary Maths Non Verbal Skills MidYIS Score

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

Average 114.3 118.7 110.8 109.0 116.9

St. error 1.3 1.4 1.3 1.3 1.4

Sex Cust

M 143 A 129 A 120 A 126 A 138 AM 133 A 137 A 144 A 146 A 137 AM 136 A 129 A 105 B 145 A 134 AM 132 A 132 A 132 A 130 A 134 AM 131 A 132 A 106 B 123 A 133 AM 115 A 149 A 110 A 134 A 132 AM 125 A 134 A 123 A 140 A 131 AM 125 A 134 A 102 B 110 A 130 AM 124 A 135 A 112 A 130 A 130 AM 118 A 142 A 143 A 111 A 130 AM 126 A 132 A 106 B 96 C 130 A

• This slide shows data for the top year 9 students. Assess the information that this data provides to help the teacher?

• Who do you think is on this list?

Vocabulary Maths Non Verbal Skills MidYIS Score

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

Average 114.3 118.7 110.8 109.0 116.9

St. error 1.3 1.4 1.3 1.3 1.4

Surname Forename Sex Cust

SALE OLIVER M 115 A 149 A 110 A 134 A 132 ADU VAL DE BEAULIEUTHOMAS M 100 C 143 A 112 A 95 C 120 AROBSON HARRY M 118 A 142 A 143 A 111 A 130 ACAIRNS OSCAR M 117 A 141 A 135 A 101 B 129 AWHITAKER BEN M 119 A 139 A 126 A 129 A 129 ABRILL MAX M 133 A 137 A 144 A 146 A 137 AKENNEDY EDWARD M 120 A 137 A 105 B 101 B 129 AWADHAM THOMAS M 113 A 136 A 120 A 134 A 125 ANEWTON BEN M 115 A 136 A 120 A 106 B 126 AWATKINS GEORGE M 124 A 135 A 112 A 130 A 130 A

This slide shows data for the top year 9 students ranked by Mathematics ability.

• What potentially important information is obtained from this data to help the teacher?

• Who is on this list?

But the questions on the test have nothing at all to do with my

subject...!

Predictions

6.1 points = D

7.4 points = A*/A

Use data that:• is reliable• gives consideration to each subject• is based on a national representative sample• provides information in an accessible format• is easily understood• IS USED TO HELP INFORM PROFESSIONAL JUDGEMENT

Give due consideration to factors such as:

• Knowledge of student • Parental support/home life• Peer influences/social life• Student attitude, interest and job commitments• Expectations of staff• Department/institution ethos • Resources • The institution’s prior value added data

Individual Pupil Record MAX BRILLDate Of Birth: 16/12/95 Student Number: 110855705547

Sex: M Form: 3S/IT1

MidYIS Band Stanine PercentileStandard

ScoreVocabulary A 9 99 133

Maths A 9 99.0 137

Non-Verbal A 9 99.0 144

Skills A 9 99.0 146

MidYIS Score A 9 99.0 137

Nationally Standardised Scores with 95% Confidence Band

50

60

70

80

90

100

110

120

130

140

150

Vocabulary Maths Non-Verbal Skills MidYIS Score

Nat

ion

ally

Sta

nd

ard

ised

Sco

res

Vocabulary Maths Non-Verbal Skills MidYIS Score50

60

70

80

90

100

110

120

130

140

150

Nationally Standardised Scores with 95% Confidence Band

Nat

ion

ally

Sta

nd

ard

ised

Sco

res

  Individual Pupil Record    

CHARLES HAYDN-TAYLOR

  Date Of Birth: 06/01/96  Student

Number: 1043340992

  Sex: M   Form: 3S/IT6           

   MidYIS Band Stanine Percentile Standard

Score

  Vocabulary A 6 75 110

  Maths C 5 46 99

  Non-Verbal B 5 51 100

  Skills A 7 82 114

  MidYIS Score B 6 63 105

           

Individual Pupil Record ALEXANDER BAILEYDate Of Birth: 01/01/96 Student Number: 1043927886

Sex: M Form: 3S/IT5

MidYIS Band Stanine PercentileStandard

ScoreVocabulary A 8 96 126

Maths A 9 98 132

Non-Verbal B 6 66 106

Skills C 4 39 96

MidYIS Score A 9 98 130

Nationally Standardised Scores with 95% Confidence Band

50

60

70

80

90

100

110

120

130

140

150

Vocabulary Maths Non-Verbal Skills MidYIS Score

Nat

ion

ally

Sta

nd

ard

ised

Sco

res

D C B A0

10

20

30

40

50

60

70

80

90

100

03

16

81

Per

cen

tBand Profile Graph: all MidYIS

cohort

Current Year 11

Skills: Percent in each BandYear 9/S2 MidYIS Test 2009/2010

9

22 23

46

0

10

20

30

40

50

60

70

80

90

100

D C B A

Pe

rce

nt

Skills – Percentage in each Band

Proof Reading: Percent in each BandYear 9/S2 MidYIS Test 2009/2010

11

23 26

40

0

10

20

30

40

50

60

70

80

90

100

D C B A

Per

cen

t

This will be interpreted as a personalised prediction

U G F E D C B A A*0

5

10

15

20

25

30

35

0 0 1

2

6

21

28

26

17

GCSE ChemistryPupils with a MidYIS Score of 109 (band B) achieved the following spread of results:

Grade

Per

cen

t

U G F E D C B A A*0

5

10

15

20

25

30

35

40

45

0 0 0 02

10

21

30

37

GCSE ChemistryPupils with a MidYIS Score of 123 (band A) achieved the following spread of results:

Grade

Per

cen

t

This boy’s ‘Baseline Suggested Grade’ is grade A*

Through our progress reporting system teachers are asked to assess his current progress against this BSG and to suggest what the likely outcomeis AT THE END OF THE COURSE

The data doesn’t work for this particular student

Give due consideration to factors such as:

• Knowledge of student • Parental support/home life• Peer influences/social life• Student attitude, interest and job commitments• Expectations of staff• Department/institution ethos • Resources • The institution’s prior value added data

You’re raising false expectation – he’ll never get that result!

You’re making us accountable for guaranteeing particular grades – when the pupils don’t get them

we’ll get sacked and the school will get sued!

Tips:1. Make sure that colleagues have every

opportunity to ask questions and be prepared to go over things multiple times.

2. Don’t get too focused on the BSGs at the expense of the chances graphs.

3. Give colleagues forms of words to use to advise when the data may not work for a particular individual.

U G F E D C B A A*0

5

10

15

20

25

30

35

40

45

0 0 0 0

2

13

3335

16

GCSE EnglishPupils with a MidYIS Score of 112 (band A) achieved the following spread of results:

Grade

Per

cen

t

U G F E D C B A A*0

5

10

15

20

25

30

35

40

45

50

55

60

65

70

75

0 0 0 0 0 1

6

25

68

GCSE EnglishPupils with a MidYIS Score of 132 (band A) achieved the following spread of results:

Grade

Per

cen

t

U G F E D C B A A*0

5

10

15

20

25

30

35

40

45

50

0 0 0 0 0

4

18

3840

GCSE EnglishPupils with a MidYIS Score of 122 (band A) achieved the following spread of results:

Grade

Per

cen

t

Make sure you have planned the logistics well in advance:•Distribution of graphs•Data entry of BSGs into MIS•Process for staff to enter necessary information•Process for distributing final progress sheets

Pupils

1. Make sure they know why they are taking the test.

2. Make sure they take it seriously3. Make sure they don’t deliberately mess it up

in order to lower their BSGs!4. Be prepared to look for clear anomalies and

re-test if necessary5. Explain the chances graphs to them clearly

Parents

1. Make sure they know why they the pupils are taking the test

2. Explain the results to them3. Explain lots of times that the chances graphs and

BSGs do NOT give personalised predictions4. Ensure that they receive good quality feedback

from staff when ambers or reds are awarded5. Encourage them to ask lots of questions