Impact, measurement and funding

Jane Tinkler

RENU RESEARCH EXCELLENCE AND FUNDING28 APRIL 2015

The Impact of the Social Sciences project

• Three-year HEFCE funded project, working with the University of Leeds and Imperial College London.

• Looked at how academic work had impact on government, business and civil society.

• Book from the research: Simon Bastow, Patrick Dunleavy and Jane Tinkler (2014) The Impact of the Social Sciences: How academics and their research make a difference. London: Sage.

1. How does impact happen?

Knowledge currently in use

Knowledge not in current use

Applied knowledgeand research

Theory-based,abstract knowledgeand research

‘Ordinary knowledge’

Dynamic Knowledge Inventory…a model of impact for the humanities and the social sciences

2. What can the focus on impact show us?

2a. What we publish and how we publish

Comparing academic and external citations shows interesting differences between disciplines

Collaborative research tends to get more citations

0 1 2 3 4 5 6 or more

0

200

400

600

800

1000

1200

Co-authorship and Number of Outputs

Number of Co-authors

Nu

mb

er

of

Ou

tpu

ts

0 1 2 3 4 5 6 or more

0

1000

2000

3000

4000

5000

6000

7000

Co-authorship and Citations

Number of Co-authors

Cit

ati

on

s r

eceiv

ed

Most outputs in our dataset were single authored, but more cites went to outputs that had at least one other author

0 1 2 3 4 5 60

1

2

3

4

5

6

Academic outputs scale

Exte

rnal

vis

ibili

ty s

cale

solid middle

16%

influential9%

communicator7%

invisible25%

appliedresearcher

17%

publisher27%

Can academics both publish and be impactful?

Social media effects

2b. How our work is communicated

Academic blogging means more people read your work

Twitter is increasingly more useful for dissemination than other channels

2c. The metrics used to indicate research qualities/quantities

Citations only give you part of the picture on the use of academic research

Shared

• Popular press mentions

• Twitter retweets• Facebook likes• Pinterest shares

Downloaded

• Web views• PDF downloads• Blog readers• Podcast listens• Time spend reading

Cited

• Referenced in government, think tank or NGOs reports

• Mentioned in legal arguments

• Used as case study evidence

Engaged

• Event audience numbers

• Exhibition visits• Practitioner

networking events

Used

• Academics as members of corporate boards

• Or in government advisory positions

• As members of practitioner networks

• Paid for research

Co-Developed

• Utilised in teaching materials

• Taken up by in professional organisations

• Built on to improve any kind of performance

Discussed

• Utilised in public debate• Referenced by

journalists• Referenced in

parliamentary debate

What would impact metrics include?

Dissemination

Impact

The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment & Management

Impact• It is not feasible to assess the quality of research impact using

quantitative indicators alone;• Research impact in the REF is broadly defined, however,

quantitative data and indicators are highly specific to the type of impact concerned;

• Viewing quantitative data about impact needs to be seen in context, and is likely to require a narrative element;

• There is potential to enhance the use of quantitative data as supporting evidence within a narrative case-study-based approach to impact assessment;

• HE Funding Bodies should build on the analysis of the impact case studies from REF 2014 to develop a set of guidelines on the use of quantitative evidence of impact (cf Digital Science/KPI study);

• These guidelines should provide suggested data to evidence specific types of impact and could also include standards for the collection of data.

2d. How our work is translated

A key problem for the social sciences is the relative lack of ‘mediating middle’ that builds long-term links and identifies impacts

3. Funding and Impact

0

100

200

300

400

500

600

700

Gro

ss v

alue

add

ed (£

bn in

real

term

s ) PrimaryManufacturingUtilities and amenitiesBusiness servicesCivic services

Source: EU Klems data, 2009

67k

£4.7bn

… and a social science dominated economyWe have a STEM-dominated research culture…

Source: LSE PPG 2014

Value of mediation of social science in the UK

£24.2bn

Indirect & induced value

£4.8bn

University spending£2.7bn

External mediators410k

625k students

35k academics

Researchfunded£851m

Research staff 32k

Economic valuePeople value

Natural Systems(e.g. astrophysics, pure maths)

Human-influenced Systems(e.g. climate change)

Human-dominatedsystems (e.g. cities, markets,information systems)

SOCIAL SCIENCES

And theirintegrative

role.

Big research challenges involve ‘human-influenced’and ‘human-dominated’ systems

We did the thing that social science does best, right? Which is not to

answer a particular question, but to change the way in which people

think about what the questions are.

We can start to answer the big and deep

questions that sociologists ask but with statistical rigour of large datasets

from computer science. I think a lot of this is about finding a commonality of language which doesn’t

exist today.

For more details:

The Impact of the Social Sciences (Sage, 2014)

Maximising the Impacts of your Research: A handbook for social

scientists (2011)

Using Twitter in University Research, Teaching and

Impact Activities: A guide for academics and

researchers (2011)

Email: impactofsocialsciences@lse.ac.uk

Twitter: @lseimpactblogFacebook: Impact of Social Sciences