Students as Partners in creating our public reputation

Contents

• What are league tables for?• Student input to league tables.• How league tables are

constructed.• What do they say about us?• How should we respond?

Where do you think we should be in a university league table?

Why are they important?

• Used in recruitment decisions – maybe.• Reputation – definitely• How our students see us• Our own self-worth

What are the problems with League tables?• League tables use the most recent data sets available

at the time of publication. • There is a time-lag when using publishing data sets and

time series data.• Different statistical analysis, methodology and

weightings are used. In the different tables• There can be changes to criteria and methodology• Data sets can change• Difficult to make comparisons between subjects and

between competitors at this lower level.

More problems....

• Categorisation of courses and subjects are different. And only concentrate on FT UG provision – this University is about so much more than that

• The editorial that appears alongside each University entry is not always available to the University to update or influence prior to publication.

• Information is often not the same as we might use internally

• Newspapers love to trumpet obvious, but wildly misleading headlines

However – we are stuck with them…..

....so, let’s make sure we understand what they say about us, and try to make them work for us

Over-riding importance – Improving Student Outcomes

• We need to allow students to reach their full potential– Celebrating individual success– Maximising individual rewards– Maximising contribution to society– How do we do this in areas of low aspiration?

• Students are increasingly consumers– Access price comparisons – is there price

sensitivity?– Access to performance comparisons

Impact of student outcomes on the University

• KIS• Unistats• National Student Survey• League Tables• Which? University Guide• College and University websites• The Complete University Guide• ……etc etc

Student Outcomes in Public Information

• Inputs– Spend per student– Staff student ratios– Entry standards– Research ratings– Cost of living– Spend on services– Faculty spend

• Outputs– Number of “good”

degrees– National Student

Survey results– Retention rates– Employability

Student Input to Public Information

• National Student Survey– Measure of final year

students– Questions on

satisfaction with course, feedback, teaching

• Used in:– League tables/KIS

(external)– Portfolio performance

review (internal)– Award annual

monitoring (internal)

Ones to watch

• Complete University Guide (May)• Guardian University Guide (June) *• Times/Sunday Times Good University Guide

(August)

• * - in University strategic plan…..aim to be in top 50

What do they say about us now?

YEAR THE COMPLETE

UNIVERSITY GUIDE

THE GUARDIAN

THE TIMES THE SUNDAY TIMES

2011 80 69 77 96

2012 99 77 89 105

2013 108 96 100 107

2014 113 92 ??? ???

SU positions in Complete University Guide for each factorEntry Standards 111/124

Student Satisfaction 54/124 <- good!!

Research Assessment 111/124

Graduate Prospects 116/124

Staff student ratio 87/124

Academic Services Spend 94/124

Facilities Spend 62/124

Good Honours 103/124

Degree Completion 108/124

Overall 113th

150 200 250 300 350 400 450 500 550 600 6500

10

20

30

40

50

60

70

80

90

100

% Good Degrees (56.3) vs Entry Standards (251)

entry standard

% g

ood

degr

ees

150 200 250 300 350 400 450 500 550 600 6500

10

20

30

40

50

60

70

80

90

100

Graduate Prospects (48.9) vs Entry Standards (251)

entry standards

grad

uate

pro

spec

ts

0.5 1 1.5 2 2.5 3 3.50

200

400

600

800

1000

1200

Overall Score (453) vs Research (1.62)

research assessment

over

all s

core

To be successful in the CUG league table :

• increase the number of good degrees awarded – how?

• recruit better qualified students who are more likely to get good degrees – WP? Value added?

• with better degrees, more graduates will get graduate entry jobs

• increase research assessment scores – limit number submitted to REF

Factor Source Weighting% Satisfied with Teaching NSS 2012 10%% Satisfied overall with

courseNSS 2012 5%

Expenditure per student (FTE)

HESA data for 2010–11, and 2011–12

15%

Student:staff ratio HESA data for 2010–11, and 2011–12

15%

Career prospects 2010-11 HESA/DLHE data 15%Value added score/10 HESA data (ie the cohort who

graduated in 201215%

Average Entry Tariff typical UCAS scores of students aged 20 or under on

entry (HESA)

15%

% Satisfied with Assessment NSS 2012 10%

Guardian league table factors

2008 2009 2010 2011 2012 2013 2014

Staffordshire 74 67 55 69 77 96 9210

30

50

70

90

110

Overall Position in Guardian Guide

Ovw

eral

l Pos

itopn

2008 2009 2010 2011 2012 2013 2014

Coventry 65 75 75 87 63 46 33

Huddersfield 94 89 96 54 53 48 42

Keele NaN NaN 50 44 54 50 45

Sunderland 82 72 62 52 48 57 65

BCU 61 56 63 66 90 60 61

Plymouth 35 48 51 49 47 61 59

Derby 93 98 105 91 94 70 79

UCLAN 68 62 82 69 70 71 87

Teeside 70 81 69 79 60 78 74

Stafford-shire

74 67 55 69 77 96 92

Liverpool JM 49 109 112 109 111 98 69

Glyndwr 118 103 92 95 102 106 108

10

30

50

70

90

110

130

Comparison with Competitorspo

sition

2010 2011 2012 2013 2014

NSS Teaching (%) 81.5632889924289

80.4511278195489

81.9548872180451

83.25 86

77.5

78.5

79.5

80.5

81.5

82.5

83.5

84.5

85.5

86.5

NSS Teaching (%)

year 2010 2011 2012 2013 2014

NSS Overall (%) NaN 78 81.8181818181818

80 85 NaN

75.0

77.0

79.0

81.0

83.0

85.0

NSS Overall (%)

2010 2011 2012 2013 2014

NSS Feedback (%) 63.3357760830865 65.4 66.2674650698603 68.2634730538922 72

59.0

61.0

63.0

65.0

67.0

69.0

71.0

73.0

NSS Feedback (%)

2010 2011 2012 2013 2014

Entry Tariff 249.418065547561 235.770276900066 238.5 245.50650229837 254

227.5

232.5

237.5

242.5

247.5

252.5

257.5

Entry Tariff

2010 2011 2012 2013 2014

Career prospects (%)

65.1504051666276

63.2253202712886

56.4021995286724

55.1157580283794

48

5.0

15.0

25.0

35.0

45.0

55.0

65.0

Career prospects (%)

2010 2011 2012 2013 2014

Value added score/10

5.24246243984631

5.16113615713964

4.71583068570222

3.97697105816477

3.8

0.5

1.5

2.5

3.5

4.5

5.5

Value added score/10

Are we trying to improving position or performance?• Clearly, we can try to play the game of moving

our league table position• What we really want to do is improve our

performance in each of the key areas to make sure there is a sustainable and genuine change

What steps can we take?

Guardian criteria Suggested Action

Entry standards Review all current standard offers, are we pitching ourselves properly against competitors? Average A level scores have gone up, have our offers?

Student/staff ratio Reviewing more thoroughly the data we submit to HESADeveloping better models of SSR to identify where investment is most needed

Spend per student Reviewing more thoroughly the data we submit to HESAIdentifying capital spend neededIncreased recent spend on libraries and IT will have an impact

Guardian criteria Action

Value added Increase number of “good” degrees awarded. Reviewing all level modules with low pass rates and average marks. Reviewing degree classification rules as part of change to % calculationIdentifying through portfolio review awards with consistently poor progression and attainment

NSS teaching, assessment and feedback and overall satisfaction

Faculty action plans, and award level plansIncreased student engagement with surveySeven principles of feedbackOnline assessment and feedback project

employment Encouraging more students to complete DLHEStaffordshire Graduate – improving our students’ chances of success

Keeping track of how we are doing• Portfolio Performance Measurement

– Provides an internal review mechanisms for award performance

– Records market attractiveness– Retention, progression and “good” degrees– National Student Survey, DLHE

• Future performance measures– Value added (difficult to get the raw data)– SSR at School and subject level– REF

Conclusions

• League Tables are a necessary evil, and a part of the HE landscape

• One of our KPIs is to ‘to be amongst the top 50 institutions in The Guardian league table’

• We all have a part to play, in explaining to students and parents what they really measure,

• Central work on-going on data returns and strategy• Schools working in partnership with our students

on supporting their experience and attainment

@mikehamlyn

blogs.staffs.ac.uk/mgh1/