“ I often say that what gets measured, gets done.

Margaret Spellings

david.moran@tribalgroup.comwww.portal4isp.com ”

Knowing every student, Knowing their potential

Why should we use data in our work

Without data, you are just another person with an opinion

Andreas Schleicher. OECD, Head of Indicators and Analysis Division

Winning is a game of inches.Humphrey Walters

Performance Comparisons

S-RateA 91%

B 73%

C 82%

D 55%

E 5%

F 99.20%

Performance Comparisons

S-Rate Hospital TypeA 91% Orthopaedic

B 73% Accident and Emergency

C 82% General Surgery

D 55% Coronary Unit

E 5% Hospice

F 99.20%

Performance Comparisons

S-Rate Hospital TypeA 91% Orthopaedic

B 73% Accident and Emergency

C 82% General Surgery

D 55% Coronary Unit

E 5% Hospice

F 99.20% Maternity

First Major Principle of Fair Evaluation

What goes in affects what comes out

Performance Comparisons

S-Rate Hospital TypeA 91% Orthopaedic

B 73% Accident and Emergency

C 82% General Surgery

D 55% Coronary Unit

E 5% Hospice

F 99.20% Maternity

Performance Comparisons

S-Rate Hospital Type Good Avg PoorA 91% Orthopaedic 97% 95% 93%

B 73% Accident and Emergency

C 82% General Surgery

D 55% Coronary Unit 48% 43% 38%

E 5% Hospice          

F 99.20% Maternity          

Second Major Principle of Fair Evaluation

Essential to compare like with like

Lies, damn lies and statistics – Mark Twain

He uses statistics as a drunken man uses lampposts - for 

support rather than illumination.

(Andrew Lang)

Statistics are no substitute for judgment.(Henry Clay)

Data Availability and Data Literacy

Ineffective

Embedded

Ignored Dangerous

Low HighData Availability

Dat

a Li

tera

cy Hig

hLo

w

Data LiteracyON YOUR TABLES, DISCUSS: DO YOU HAVE

1. Sufficient data:– to enable the key questions and factors to be explored

2. Sufficient access:– to systems which enable key elements of data to be linked

3. Sufficient experience and understanding:– to find the smallest amount of data needed - and how best to present it 

4. Sufficient embedding:– such that individuals have an appropriate view about the reliability of data

5. Sufficient confidence:– to be able to justify why we are NOT doing something as well as the things 

we have decided to do6. Sufficient humility:

– to enable our own assumptions to be challenged

Terminology

C

D

A An estimateA target

B A guess Daft

Calculating what you would expect a group of pupils to achieve, based upon the progress of similar pupils last year, is ?

Terminology

CAn estimate

Calculating what you would expect a group of pupils to achieve, based upon the progress of similar pupils last year, is ?

Past knowledge = estimate

Using Estimates with StudentsYour target grade

is …I thought I could do better

How do they expect me to achieve that?

I’ll show them!

I can get that easily

If you make average progress, you might get a… Let’s look at the range of grades achieved by similar students last year …. …. what will you aim to achieve?Interesting ..

Maybe I could do that …If one in five did that last year…?

Using Estimates with Students

What factors impact upon pupil achievement

AAttitudeAttendanceAptitude

MMotherMath capabilityMovement between schools

B Behaviour N Neighbourhood

CCrimeChallengeComputer access

OOpportunity

DDataDadDependents

PPrior attainmentPolicy

EEngagementEnglishEnvironment

QQuestioning abilityQuality of provision

F FoodFamily R Reading

GGender

SSchoolSpecial needsSocial welfare

H Home life T Teacher qualityTest ability

I Importance of EducationIntervention U Understanding

UniformJ Job aspirations V Variation

K Knowledge W Willingness to learnWriting

L Lifestyle XYZ

What factors impact upon pupil achievement

16.678 + 0.0054*(KS1 APS squared) + 0.672 *KS1 APS + 0.033*(KS1 reading points - KS1 APS) + 0.271*(KS1 maths points - KS1 APS) + 0.2750 (if in care) - 0.681*IDACI score - 1.528 (if School Action) - 2.437 (if Action Plus or Statemented) - 0.509 (if joined at start of or during Y6) - 0.306 (if joined at start of or during Y5) - 0.227 (if joined at start of or during years 3 or 4) - 0.272 (if female) - 0.626*(age within year where 1 Sept= 1.00, 31 Aug = 0.00)+ for EAL pupils only (2.173 + 0.0036*(KS1 APS squared) - 0.1762 *KS1 APS )+ ethnicity coefficient+ for FSM pupils only ( - 0.327 + FSM/ethnicity interaction)

UK KS2 to KS4 CVA

Simple Value added

Time

Ach

ieve

men

t

Time

Ach

ieve

men

t

Better than average = Positive Value Added

Lower than average = Negative Value Added

In the UK, we take 589,000 pupils and look at the average of what happened

KS2 APS KS4 APS

Different models = different estimates

Time

Atta

inm

ent

Different characteristics are used in complex mathematical models to create estimates based on a number of characteristics... Different estimates are created.

Differences• If two assessments are different

– One might be wrong

– They might BOTH be wrong

– They might be assessing different things

TriangulationAnalysis A Analysis B

Teachers Professional JudgementBasis for action Investigate Further

Check Accuracy Challenge Assumptions

UK GCSE outcomes at age 16

What would/could this look like for Nashville?

What would the input variable be?

What would the output variable

be?

35.935.835.735 .635.535 .435.335.235.135 .034.934 .834 .734.634.534 .434.334.234 .134.033.933 .833 .733.633.533 .433.333.233 .133.032.932 .832 .732.632 .532.432.332 .232.132.031 .931.831.731 .631 .531.431 .331.231.131 .030.930.830 .730 .630.530.430 .330.230.130 .029.929 .829.729.629 .529.429.329 .229.129 .028 .928.828.728 .628.528.428 .328.228.128 .027 .927.827.727 .627.527.427 .327.227.127 .026 .926.826 .726.626.526.426 .326.226 .126.025.925.825 .725.625 .525.425.325.225 .125.024 .924 .824.724.624 .524.424.324 .224.124.023 .923 .823.723.623 .523.423 .323.223.123 .022.922 .822.722 .622.522.422 .322.222.122 .021.921 .821.721 .621.521 .421.321.221 .121.020.920 .820.720.620 .520.420 .320 .220.120.019 .919.819.719 .619.519.419 .319.219 .119 .018.918.818 .718.618.518 .418.318.218 .118 .017.917 .817.717.617.517 .417.317 .217.117 .016.916 .816.716.616 .516.416.316 .216.116.015.915 .815.715 .615.515.415 .315.215.115 .0

650060005500500045004000350030002500200015001000500

0

Pupils' Key Stage 2 Points

No's

of P

upils

National distribution of Key Stage 2 Pointsfor 600,000 pupils annually

12 18 24 30 36

Low Below Avge Above High

What would/could this look like for Nashville?

Low Below Avg Above High

8% 26% 57% 83% 95%

The Wensleydale SchoolA Specialist Science College

Richmond Road, Leyburn North Yorkshire DL8 5HY

www.wensleydale.n-yorks.sch.uk

What might you do to exceed average?

The only judgements that can be made…

Mainstream secondary schools ranked

UCI

LCI

Statistically Average Statistically AboveStatistically Below

UCI

LCI

UCI

LCI

Always check if the confidence intervals cross the magical 1000 median?