Key issues from yesterday and for panel

Lorraine Dearden

Dearden

• School League tables here to stay in England

• But very difficult to measure the value added by schools

• Given not likely to go away, we need to do it better– To inform parent choice– To make schools accountable

Marsh et. al.

• Variation of outcomes explained by schools/universities not great

• Variation in outcomes explained by teachers within schools/universities where focus should be (like US)

• Should focus more on teachers who make much more of a difference for educational outcomes

• Wider measures of outcomes

Cunha

• Can think of measuring school effectiveness in way you think about skill accumulation over the life-cycle

• Really important to think of implications of measurement error and sampling error

• Important to remember that academic outcomes are not the only important measure of school effectiveness

• Transparency is probably bad...

Leckie and Goldstein

• School league tables ignore the uncertainty in using current performance as a guide to future performance– Adjusting for this uncertainty reduces the number of

schools that can be separated to almost none

• The way you should measure school effectiveness for parents and schools accountability different

Vignoles, Micklewright and Dearden

• Schools are differentially effective across the prior ability distribution – Having average school measures of effectiveness may

not be useful– parents need to know the value added by a school for

children with similar prior attainment to their own child

– These measure needs to be transparent and simple• Rankings of school vary quite widely if look within

different prior ability groups

Brown and Tzavidis

• Use M-quantile approaches to estimate ‘robust’ CVA models – Measures how well a child does relative to

children with same inputs and prior ability (cf typical regression approaches)

• If rank LAs using this efficiency measure in London get very different story to using standard CVA regression approach

Smith• There are many dimensions to measuring

school effectiveness • Mixture of – Regulation– Professionalism– Markets– Performance management (teacher work force

data)• Hard to do well and we are doing alright...– The perfect and the good: friends or enemies

Key issues

• Measuring school effectiveness very hard• Easy to critisize CVA models, but the fact that

the government has adopted a CVA model and gives researchers access to NPD should not be under-estimated

• Remember Jeff’s wise words!!! We have incredibly rich vein of research in the UK using these data and we have learned much...

Where does research need to go?

• Teacher effects in the UK data but we cannot ignore schools. How big is the school effect?

• Making full use of teacher work force data which links teachers to schools and has some info on characteristics (out this summer)

• Modelling parent preferences– Local information, statistical information– How actual decisions are made?

Where does research need to go?

• Measurement error– Validation with external sources

• Sampling error – Need better information on who applies to schools?

• Stability over time? Significant differences between schools?

• Are all schools really equally as effective when projecting forward 6 years? Or is it just we cannot be certain?– Pooling data?

Where does research need to go?

• Has UK research underestimated the importance of strategic responses by agents?

• How serious is the problem? This one for more research – how do teachers/schools game? – How do parents game?– study curriculum choices etc?

Where does public policy go?

• Clearer recognition we have two sets of data or measures that we need– Measure for accountability (of schools? Of teachers?)– Measures for parents (simple is an issue!)– Maybe approach for each is different

• Recognition it is a mix of– Regulation– Professionalism– Markets– Performance management (teacher work force data)

Where does public policy go?

• Transparent? Not transparent?– Gaming the system. – Maybe not transparent for accountability but

transparent for parents?– If gaming is at KS2 is that a problem?– If the outcome measure is outside school control

is it a problem?

Where does public policy go?

• Wider set of outcomes– Non cognitive?– HE participation?– Behaviour e.g. Crime, exclusions– Safety (this is something parents care about)

• Multiple measures to minimise gaming• But if measures are within school (not

externally validated) is this a problem?

Leave this to the panel discussion..