GSOE Impact WorkshopImpact and the REF

19th May 2010

Lesley Dinsdale

Key features of the REF

• Hefce announcement on initial decisions 25th March 2010: • Essentially a similar overall exercise as the RAE:• Peer review is fundamental• Institutional selection of staff and best four outputs to continue• Institutional submission policies• Codes of Practice: selection criteria, allowances for individual

circumstances

• But assessment of esteem has been replaced by assessment of the impact of research

REF Structure of submissions

Outputs Environment ImpactElement:

Assessedby:

Expert review, possibly informed by bibliometrics in some sub panels

‘RA5 type’ narrative

Supported by indicators: income, students

Narrative, case study approach giving exemplars

Supported by indicators

Profile against criteria: originality, significance, rigour

Reported as:

Profile . Criteria relating to outputs and impact

Profile against criteria: reach and significance

Individual, of staff submitted

The UOA The UOA

Weightings %? 60/70? 15/20? 10/20?

Assessment of Impact: Proposals

• Assessed at the unit level, not individual

• Definition of impact for REF is wide but does not include academic impact

• Majority of assessment by case studies (1 per 10 FTE?)

• Supported by overarching impact statement

Assessment of Impact: Proposals

• Criteria:• Impact ‘occurred’ during survey period (Jan 2008 – survey date)• Must relate to underpinning research (from date?) of a ‘sufficient

standard of rigour and originality’• Research may be a specific piece or a ‘body of work’• Contribution by the institution to both the underpinning research

and impact

• Impacts must be evidenced, including ‘indicators’ where appropriate: potentially auditable

• Assessment criteria: reach and significance (‘breadth’ and ‘depth’)

Impact: Hefce pilot

• To test Hefce proposals for methodology – submissions and assessment criteria

• 29 universities, 5 UOAs including Social Policy and Social Work (UOA40)

• ‘Mock’ impact submissions – case studies and impact statement, March 2010

• Pilot assessment panels – 50/50 academic/users of research. Govt Chief Social Scientist Chair of UOA40

• Panels to provide detailed report in Autumn 2010

• Impact: UOB pilot experience and issues

• High (but not universal) buy-in from academic participants

• Submissions led by UOA Academic Coordinators with a small team – their detailed local knowledge and energy/enthusiasm was vital

• Identification of case studies (12 submitted in total) was local – no systematic corporate sources

Impact: UOB pilot experience and issues

• Evidencing impact:• Can we demonstrate that peoples’ lives have been

changed in some way?• Some of the Hefce proposed indicators of impact are

activities rather than impacts...• ...new concept of ‘intermediate impact’• Hefce do not intend to make this an impossible

exercise!

Impact: UOB pilot experience and issues

• Many highly subject-specific issues :• Particularly in the social sciences, there is not a linear

progression from research to impact• Govt commissioned work sometimes not published in a

peer reviewed form?• What about high-quality research findings that are

rejected by policy makers for political reasons?• What about ‘negative’ research findings?