Analysis of the Research Excellence Framework 2014 impact case studies

Ian Thornton

In partnership with

King’s analysis

Four strands to extract insights

Topic modelling: Identify hidden thematic structures or topics in corpus of documents

Keyword in context: Identify keywords displayed within surrounding context

Information extraction: Automate extraction of specific words (nouns) such as countries

Qualitative analysis: Read and hand-code samples of case studies

287 relevant case studies, found through two methods

Surprisingly little overlap between case studies identified by the two methods

These 287 case studies were mostly submitted to the life sciences and social sciences panels

The research has impact in 175 countries

With many usual suspects, but also c.40 mentions for Australia and the Netherlands

This research was supported by a very wide range of UK organisations

But the number of mentions did not

correlate at all with investment in development-

relevant research

2013-14£383m of public R&D funding is spent on internationaldevelopment.

Equal to 8.3% of the £4.6bn science budget

% of UK public R&D funding spent on international development

Breakdown by UKCDS members2013-14 Each £ symbol = £8mDFID

£ £ £ £ £ £ £ £ £ £ £ £ £ £ £ £ £ £ £ £ ££ £ £ £ £ £ £ £ £ £ £ £ £ £ £ £ £ £ £ £ £MRC

£ £ £ £ £Other RCs

£Wellcome Trust

£ £ £ £4.6% of Wellcome Trust committed awards are on science for international development

2011-12 2012-13 2013-140%

5%

10%

Given that they spend dramatically different amounts (which itself is changing)

Most research claimed to have impact across multiple sectorsWith lots claiming to have an impact on health outcomes

The majority of case studies claiming to

impact on health were not submitted to life

sciences UoAs

84 case studies cited observed outcomes stemming from the research

Caveats galore, but still (I think) useful insights

• Caveats with the sample

• Caveats with the methods

• Hints of interdisciplinarity – but the UKCDS Board decided not to pursue

What people really want are stories, not analysis

UKCDS’ top 20