HOW CONCENTRATED IS THE UK RESEARCH BASE?THE DISTRIBUTION OF EXCELLENCE AND DIVERSITY

JONATHAN ADAMS

14 OCTOBER 2009

Previously among studies by Evidence• HEFCs/UUK – Maintaining Research Excellence

– Recognition of a peak supported by a research platform– Evolution of 3 to 4 to 5 grade units

• HEFCE – Role of QR funding– Strategic significance of the block grant

• Universities UK – Excellence and Diversity– Regional network of research competence– Importance of an evidence base for research policy

• OST/DIUS/BIS– Diversity as a critical element in research policy

RAE2008 – a novel outcome with dispersed rewards• Gareth Roberts’ proposals on research profiling

– Problem of the ‘cliff edge’– Recognising dynamic excellence in the research base

• Unexpected differences between panels– The problem of interpreting methodology and standards– The absence of a commonly understood standard?– The dispersed map of reward-able research

• Decisions about funding allocations– 2* = 1 (x 3 to) 3* = 3 (x 2.33 to) 4* = 7– Greater relative rewards for modest gains than for very

expensive world-class excellence (cf 2001)

How can we explore the spread of excellence?• We have

– Panel outcomes from RAE2008 – but opinions differ, so ...– Bibliometric data on a consistent basis from 1981

• We split the HE sector into crude groups– All HEIs– Russell Group– 1994 Group– Universities with a shorter history of research investment

• And we created an elite ‘golden triangle’ band– Oxford (C12th), Cambridge (1209), UCL (1836), Imperial

(C19th) , LSE (1895)

Excellence is linked to selectivity

15

20

25

1.1

1.2

1.3

1.4

1981 1986 1991 1996 2001 2006

Share (%) of UK research output co-authored by golden triangle

Rela

tive

impa

ct o

f UK

rese

arch

pub

licati

ons

Papers co-authored by five leading institutions

UK citation impact

5 year movingaverage

1989 1992

And excellence is concentrated

UK output Total articles and reviews, 2002-2006

Papers above world average impact

Papers with impact exceeding four times world average

Papers with at least one co-author from the HE research base

306,661112,358

37% of HE total

16,060

5.2% of HE total

Papers with at least one co-author from the Russell Group

204,307

66% of UK HEIs’ total

79,505

70% of HE output

39% of Russell total

11,734

73% of HE output

5.7% of Russell total

Papers with at least one co-author from a ’golden triangle’ HEI

87,157

28% of HEIs’ total

37,370

33% of HE output

43% of GT total

6,308

39% of HE output

7.2% of GT total

The problem with simplistic indicators• They don’t really express the complexity of

research performance• Average impact (e.g. ‘crown indicator’) can be very

misleading– Research Council studies reveal error of interpretation– Skewed data, median much smaller than average– Lots of papers are not cited– The interesting bit is about how much is really, really cited

lots

• So we we prefer Impact Profiles®

UK background and ‘golden triangle’

0

5

10

15

20

25

uncited RBI > 0 < 0.125 RBI ≥ 0.125 < 0.25

RBI ≥ 0.25 < 0.5 RBI ≥ 0.5 < 1 RBI ≥ 1 < 2 RBI ≥ 2 < 4 RBI ≥ 4 < 8 RBI ≥ 8

Perc

enta

ge o

f out

put

2002

-20

06

UK higher education sector, all research fields - 306661 papers UK 'golden triangle', all research fields - 87157 papers

This is the small but critical excess of really highly cited research output

Even smaller differences separate the UK and USA research profiles

0

5

10

15

20

25

30

RBI = 0 RBI > 0 - 0.125 RBI 0.125 - 0.25 RBI 0.25 - 0.5 RBI 0.5 - 1 RBI 1 - 2 RBI 2 - 4 RBI 4 - 8 RBI > 8

Perc

enta

ge o

f out

put 1

996-

2005

UK 10 Years - 686476 PapersUSA 10 Years - 2888315 Papers

Excellence extends to other institutions

0

5

10

15

20

uncited RBI > 0 < 0.125 RBI ≥ 0.125 < 0.25

RBI ≥ 0.25 < 0.5 RBI ≥ 0.5 < 1 RBI ≥ 1 < 2 RBI ≥ 2 < 4 RBI ≥ 4 < 8 RBI ≥ 8

Perc

enta

ge o

f out

put

2002

-200

6

UK higher education sector, all research fields - 306661 papers Russell Group, all research fields - 204307 papers

UK 'golden triangle', all research fields - 87157 papers

But the differences become very fine

0

5

10

15

20

uncited RBI > 0 < 0.125 RBI ≥ 0.125 < 0.25

RBI ≥ 0.25 < 0.5 RBI ≥ 0.5 < 1 RBI ≥ 1 < 2 RBI ≥ 2 < 4 RBI ≥ 4 < 8 RBI ≥ 8

Perc

enta

ge o

f out

put

2002

-200

6

UK higher education sector, all research fields - 306661 papers Russell Group, all research fields - 204307 papers1994 Group, all research fields - 58083 papers

And if we distil further, other complexity is revealed

0

5

10

15

20

uncited RBI > 0 < 0.125 RBI ≥ 0.125 < 0.25

RBI ≥ 0.25 < 0.5 RBI ≥ 0.5 < 1 RBI ≥ 1 < 2 RBI ≥ 2 < 4 RBI ≥ 4 < 8 RBI ≥ 8

Perc

enta

ge o

f out

put

2002

-200

6

Russell-5, all research fields - 126885 papers 1994 Group, all research fields - 58083 papers

UK 'golden triangle', all research fields - 87157 papers

Older institutions have no monopoly on quality, but the peak is distinctive

0

2

4

6

uncited RBI > 0 < 0.125 RBI ≥ 0.125 < 0.25 RBI ≥ 0.25 < 0.5 RBI ≥ 0.5 < 1 RBI ≥ 1 < 2 RBI ≥ 2 < 4 RBI ≥ 4 < 8 RBI ≥ 8

Perc

enta

ge o

f out

put 2

002

-200

6

UK higher education sector - 306661 papers 1994 Group, all research fields - 58083 papers

Russell-5, all research fields - 126885 papers UK 'golden triangle', all research fields - 87157 papers

For the 1994 Group, there are relatively fewer low-cited papers and relatively more high-cited papers than for many in the Russell Group

What does this imply?• There is a very concentrated peak of exceptional

excellence• UK international competitiveness is associated with

selectivity that increased concentration• There is a regional network of exceptional quality,

much in dynamic institutions created in the 1960s• The balance that supports diversity & dynamism

and the concentrated support of international excellence is very fine

• Model first, meddle second