Delivering Strength Across the Piece

David SweeneyDirector, Research, Education and Knowledge Exchange

HEPI, Royal Society31 March 2015

How it worksREF assesses the quality of

research in all UK universities, in all

disciplines. It is carried out by 36 expert panels, grouped

into 4 main panels.

2011-12

Preparation

Panels were appointed.Guidance and criteria were published.

2012-13

Submissions

Universities made submissions in whichever subjects they chose to.

2014

Assessment

36 expert panels reviewed the submissions, guided by the 4 main panels.

Main Panel A: Medical and life sciences

Main Panel B: Physical sciences and engineering

Main panel C: Social sciences

Main Panel D: Arts and humanities

What was assessed

Panels judged the overall quality of each submission

Quality of research outputs

Impact of research on society

The research environment

65% 20% 15%

191,150 research outputs by 52,061 staff were reviewed

6,975 impact case studies were reviewed

The review was based on data and information about the environment

World-leading quality was found in diverse submissions and institutions across the UK

-10%

0%

10%

20%

30%

40%

50%

60%

Average % 4* for each institution

Number of staff submitted

• Submissions from the 154 institutions ranged from 3 staff in a single subject to over 2,500 staff in 32 subjects

Three-quarters have at least 10 per cent of their work graded as world-leading (4*).

The top quarter have at least 30 per cent of their work graded as world-leading (4*).

UK research – Punching beyond its weight

World shares data derived from Elsevier - International Comparative Performance of the UK Research Base – 2013

Population

Research

ersGERD

Article

s

Downlaods

Citations

Highly

cited arti

cles

Patents0

2

4

6

8

10

12

14

16

18

0.9%

4.1%3.2%

6.4%

9.5%

11.6%

15.9%

1.8%

Research Excellence Framework 2014• 22% of outputs were judged world-leading (4*), up from 14% in

the 2008 RAE

• 50% were judged internationally excellent (3*), up from 37% in the 2008 RAE

• This confirms independent evidence of UK research performance

The assessment celebrates the diversity of staff and excellent research of all types

• New arrangements were made for early career researchers and staff with other circumstances:− 28% were submitted with circumstances, up from 13% in the RAE

− Their outputs were found to be of equally high quality

• Excellence was found across all forms of research including applied, practice-based, basic and strategic research:− Interdisciplinary research was found to be of equally high quality

− Most impact case studies drew on research from multiple fields

− At least a quarter of journal papers involved collaboration across UK universities, and many involved overseas collaboration

Outputs by early career researchers and staff with other circumstances were found to be of equally high quality to other staff

All staff (188,550 outputs)

Staff with any circumstances (27,231 outputs)

Staff with ECR as a circumstance (16,574 outputs)

Staff with other circumstances (13,624 outputs)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Profile for outputs by staff with circumstances

%4*%3*%2*%1*%U

Outputs identified by institutions as interdisciplinary* were graded equally to other outputs

* Note that these represent a small proportion of interdisciplinary work that was submitted

All outputs (35,056)

Outputs flagged as interdisciplinary (4,042)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Profile for outputs flagged as interdisciplinary,for 31 HEIs that flagged >5% of their outputs

4*3*2*1*U

For the first time, REF has demonstrated the impact of UK research in all subjects

• Over 250 research users judged the impacts, jointly with academic panel members.

• 44% of impacts were judged outstanding (4*). A further 40% were judged very considerable (3*).

• Impressive impacts were found from research in all subjects.

• REF shows many ways in which research has fuelled economic prosperity, influenced public policy and services, enhanced communities and civic society, enriched cultural life, improved health and wellbeing, and tackled environmental challenges.

Excellence was found across institutions with submissions of all sizes

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Overall quality profile: average 4* by institu-tion

Submitted Cat A FTE staff

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Overall quality profile: average 3*+4* by institution

Submitted Cat A FTE staff

Impressive impact was found across institutions with submissions of all sizes

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Impact: average 4* by institution

Submitted Cat A FTE staff

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Impact: average 3*+4* by institution

Submitted Cat A FTE staff

Exceptionally strong environments were found across institutions with submissions of all sizes

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Environment: average 4* by institution

Submitted Cat A FTE staff

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Environment: average 3*+4* by institution

Submitted Cat A FTE staff

Average profiles by main panel

Overall

Outputs

Impact

Environment

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%Main Panel A average profiles

Overall

Outputs

Impact

Environment

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%Main Panel B average profiles

Overall

Outputs

Impact

Environment

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%Main Panel C average profiles

Overall

Outputs

Impact

Environment

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

Main Panel D average profiles

4* 3* 2* 1* U/C

The assessment produced more consistent results across subjects than in the 2008 RAE

MPD

MPC

MPB

MPA

0 5 10 15 20 25 30 35 40 45

Range of UOA average 4*

RAE maximum 4* RAE minumum 4*REF maximum 4* REF minumum 4*

MPD

MPC

MPB

MPA

0 10 20 30 40 50 60 70 80 90 100

Range of UOA average 3*+4*

RAE maximum 3*+4* RAE minumum 3*+4*REF maximum 3*+4* REF minumum 3*+4*

(Parts of) Books (28,451)

Journal articles (157,147)

Physical artefacts (755)

Exhibitions and performances (1,743)

Other documents (1,685)

Digital artefacts (760)

Other (554)

All output types (191,095)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Profile for output by type

4*3*2*1*U

Excellence was found in all types of research output

Research income increased since the RAE in all main panels and from almost all sources

Main Panel A (43% increase)

Main Panel B (29% increase)

Main Panel C (15% increase)

Main Panel D (44% increase)

0 200 400 600 800 1,000 1,200 1,400 1,600 1,800 2,000 2,200

Annual average research income reported to HESA (£Millions in 2012-13 prices)

REF period: Aver-age annual income

RAE period: Aver-age annual income

Research Councils (42% increase)

UK government (30% increase)

UK charities (15% increase)

UK industry (2% decrease)

EU government (90% increase)

EU other (96% increase)

Non-EU (73% increase)

Other (30% decrease)

0 200 400 600 800 1000 1200 1400 1600

2001-02

2002-03

2003-04

2004-05

2005-06

2006-07

2007-08

2008-09

2009-10

2010-11

2011-12

2012-13 0

100

200

300

400

500

600

700

800

Main Panel A subjects

2001-02

2002-03

2003-04

2004-05

2005-06

2006-07

2007-08

2008-09

2009-10

2010-11

2011-12

2012-13 0

200

400

600

800

1000

1200

Main Panel B subjects

Research Councils

UK charities

UK government

UK industry

EU government

EU other

Non-EU

Other

2001-02

2002-03

2003-04

2004-05

2005-06

2006-07

2007-08

2008-09

2009-10

2010-11

2011-12

2012-13 0

20

40

60

80

100

120

140

160

180

Main Panel C subjects

2001-02

2002-03

2003-04

2004-05

2005-06

2006-07

2007-08

2008-09

2009-10

2010-11

2011-12

2012-13 0

10

20

30

40

50

60

70

80

Main Panel D subjects

Research Councils

UK charities

UK government

UK industry

EU government

EU other

Non-EU

Other

Income sources vary by main panel

World-leading research

• Impressive impacts were found from research in all subjects

• Importance of peer review in research

• Multi-disciplinary research and collaboration

• Research on design of technology for gamma-ray detection influencing new counter-terrorism practices

• Technological advances from research critical in fields of homeland security and nuclear safety

• Spin-out company from the research, Symetrica, recognised as example of best practice

Advances in gamma-ray detection and imaging: University of Southampton

• Research into engine technologies resulted in a major new partnership with engineering multinational BorgWarner

• Collaboration involves multi-million-pound investment, as well as significant job creation and safeguarding

• Substantial contract with Jaguar Land Rover, furthering its R&D activities and upskilling its workforce for the benefit of the UK automotive supply chain and the local and national economy

Collaborative Strategic partnership with BorgWarner: University of Huddersfield

• First major piece of published research on young adult carers

• Led to recognition of a ‘new’ group of carers in policy and practice, and the development of two new psychometric instruments

• Raising awareness has resulted in provision of new services and support and further education provision for young adult carers across the UK

Bringing recognition and improved welfare service provision to young adult carers: University of Nottingham

• A four-year research project exploring the impact of the cold war on modern art, architecture and design

• Provided new interpretations of the history of design in the second half of the 20th century for a wide public, international audience

• Impact in the fields of public and media understanding of post-Second World War cultural history and on the curatorial and collecting activities of museums around the world

‘Cold War Modern’: the international impact of modern design on public understanding and curatorial practice: Royal College of Art

World-leading research

• Impressive impacts were found from research in all subjects

• Importance of peer review in research

• Multi-disciplinary research and collaboration

Thank you for listeningd.sweeney@hefce.ac.uk