dr steven hill, head of research, higher education funding council for england
TRANSCRIPT
![Page 1: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/1.jpg)
Understanding the impact of the Research Excellence Framework
Steven Hill
Head of Research Policy
Science Summit 2014
03 December 2014
![Page 2: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/2.jpg)
• Introduction to the Research Excellence Framework (REF)
• Results publication and analysis
• Analysis of REF submissions
• Evaluation of the REF
• Future REF
Summary
![Page 3: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/3.jpg)
• Introduction to the Research Excellence Framework (REF)
• Results publication and analysis
• Analysis of REF submissions
• Evaluation of the REF
• Future REF
Summary
![Page 4: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/4.jpg)
Overall process
• Universities makes submissions
• Submissions in 36 discipline-based Units of
Assessment
• Universities select which eligible staff to include in
submissions
• Submissions are assessed by sub-panels, one for
each Unit of Assessment
• Sub-panels are grouped into 4 main panels,
responsible for consistency
• Results provided as quality profiles
Overview:
![Page 5: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/5.jpg)
The assessment framework
Overall quality
Outputs
Maximum of 4 outputs per researcher
Impact
Impact template and case studies
Environment
Environment data and template
65% 20% 15%
Overview:
![Page 6: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/6.jpg)
• Introduction to the Research Excellence Framework (REF)
• Results publication and analysis
• Analysis of REF submissions
• Evaluation of the REF
• Future REF
Summary
![Page 7: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/7.jpg)
• 18 December 2014: Results day!
• Overall quality profiles for each HEI/UoA
• And sub-profiles for outputs, impact and environment
• Analysis of results and process
• [Eligible staff numbers from HESA]
• Early January 2015
• Publication of submissions
• February 2015
• Panel and sub-panel overview reports
• REF Manager’s report
The REF results and other outcomes
![Page 8: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/8.jpg)
• Introduction to the Research Excellence Framework (REF)
• Results publication and analysis
• Analysis of REF submissions
• Evaluation of the REF
• Future REF
Summary
![Page 9: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/9.jpg)
Analysis of REF Impact case studies
![Page 10: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/10.jpg)
6,975
![Page 11: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/11.jpg)
• Work commissioned from Digital Science and Kings College London
• Two objectives:
• Publish case studies as a searchable, text-minable database
• High level synthetic analysis of case studies – what do they tell us about the impact of research?
• Published at end of March 2015
• Crucial evidence for spending review
• Database will lead to further detailed analysis
Analysis of REF Impact case studies
![Page 12: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/12.jpg)
• What are the features of Units of Assessment that have scored very highly in the REF?
• Funding? Talent management? Strategic focus? Multi- and inter-disciplinary research?
• Does this vary by HEI type? Discipline?
• Commissioning shortly
• Publication in mid 2015
Factors leading to high performance
![Page 13: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/13.jpg)
• Introduction to the Research Excellence Framework (REF)
• Results publication and analysis
• Analysis of REF submissions
• Evaluation of the REF
• Future REF
Summary
![Page 14: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/14.jpg)
• Work commissioned from RAND Europe
• Phase 1 – evaluation of preparation of submissions
• Challenges, benefits and costs to HEIs
• Sample of 21 HEIs (35% of case studies)
• Phase 2 – evaluation of assessment process
• Strengths and weaknesses of the impact assessment process
• Panel members: survey, focus groups, interviews
• Publication late March 2015
Evaluation of impact element
![Page 15: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/15.jpg)
• Work commissioned from Technopolis
• Estimate of costs to sector of the REF
• Specific focus on ‘individual staff circumstances’
• Benefits of REF to institutions
• Comparison to equivalent study following RAE2008
• [Impact element covered in impact evaluation]
• Publication late March 2015
Review of costs and benefits
![Page 16: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/16.jpg)
• Written feedback sought in early 2014
• Responses from more than 100 HEIs
• Headline messages:
• Benchmarking, driver of quality
• Some aspects burdensome for institutions
• Positive effects of new staff circumstances measures
• Many positive views about impact (but issues with the detail)
• Publication of full analysis late March 2015
Feedback from participating HEIs
![Page 17: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/17.jpg)
• Conferences and focus groups November 2014/January 2015
• User panel members
• Academic panel members (x2)
• Covering all aspects of REF (including contribution to impact evaluation and metrics review)
• Publication late March 2015
Feedback from panel members
![Page 18: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/18.jpg)
• Introduction to the Research Excellence Framework (REF)
• Results publication and analysis
• Analysis of REF submissions
• Evaluation of the REF
• Future REF
Summary
![Page 19: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/19.jpg)
• Independent panel chaired by Professor James Wilsdon
• Call for evidence – summary published
• 57% ‘sceptical’ about use of metrics
• Literature review
• Feedback from REF panel members
• Workshops
• ‘In metrics we trust?’
• ‘Metrics for all?’
• Metrics in the Arts and Humanities
• Emerging findings March 2015, publication Summer 2015
Review of metrics and research assessment
![Page 20: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/20.jpg)
• Request from minister
• Informal consultation with
• Overseas stakeholders
• UK stakeholders (roundtable event)
• Open survey
• Advice to minister in next few weeks
International REF
![Page 21: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/21.jpg)
• Most evaluation evidence complete March 2015
• Policy development/informal consultations Summer 2015
• Proposals and formal consultation Autumn 2015
Future REF – tentative timetable
![Page 22: Dr Steven Hill, Head of Research, Higher Education Funding Council for England](https://reader030.vdocuments.us/reader030/viewer/2022032419/55a375ee1a28abf8238b4597/html5/thumbnails/22.jpg)
• Introduction to the Research Excellence Framework (REF)
• Results publication and analysis
• Analysis of REF submissions
• Evaluation of the REF
• Future REF
Summary