public engagement with research masterclass 10: some ideas .../file/ideas_for... · costing your...

Public Engagement with Research

Masterclass 10: Some ideas for evaluating your public engagement activity

Fran Marshall, Research and Evaluation Manager

Suzanne Spicer, University of Manchester

20/01/2014 © The University of Sheffield

Outline of session

2pm Welcome

2.10pm Fran Marshall

2.45pm Suzanne Spicer

3.20pm Round up and questions

3.30-4.00pm Mince pies and evaluation

Introduction to the project and Public Engagement with Research team

• Introduction to the Public Engagement with Research Team

• The Masterclass series

• What we can do to help?


Masterclass series

Masterclass title Date & time

What to think about when organising your public engagement event

27th February

How to identify partners in the city? Internal and external 27th March

Public Engagement or Widening Participation/ Outreach? Organising for children and young people – what you need consider

24th April

Relating Public Engagement to Impact 29th May

Costing your Public Engagement activity and how the University finance systems work

26th June

Film & documentary skills 31st July

Writing a funding bid and identifying potential funds 26th September

Writing a risk assessment for your event 30th October

Media and social media training 20th November

Ideas for evaluating your public engagement event 18th December


Who is here?

• About the audience – show of hands

• Who has organised a public event before? Who has organised 5 or more?

This is an inclusive environment to come together to learn and share experiences of public engagement


• The University’s Good Research and Innovation Practices policy

www.sheffield.ac.uk/ris/grip

• Minimum accepted practices in public engagement

• Higher practices to aspire to


Fran Marshall

Research and Evaluation Manager

What is Public Engagement?

• “Public engagement describes the myriad of ways in which the activity and benefits of higher education and research can be shared with the public. Engagement is by definition a two-way process, involving interaction and listening, with the goal of generating mutual benefit ”

National Coordinating Centre for Public Engagement

Who are your ‘public’?

• Who are you expecting to attend your event?


• Design the evaluation activity around your audience

Resources • Who will do your evaluation on the day?

• How much time do you have to analyse the data?

• What do you want to do with the data?

• How much money do you have for evaluation?


Monitoring or evaluation?

• Monitoring is counting

• Audience numbers, demographic information

• Evaluation is about critical reflection

• Can be: formative, about a process or summative


Types of evaluation

• Formative: to support your activity, e.g. identifying gaps or barriers, gathering information to identify the need for your activity. Testing ideas and concepts – what is going to be most effective? What works best?

• Process evaluation: gathering information about how you can do it better next time

• Summative: i.e. a summary – final impact of your activity

• Is it a single evaluation or do you want base line and comparative data?


What do we need to evaluate? • Simple answer is the

public engagement activity you are doing

• More complex answer is what can we evaluate?

• What do you want to know v what can the audience tell you?


Why do we need to evaluate? • assistance with planning

• provides evidence

• demonstrates value for money

• reporting to funders

• learning that can be shared with others and inform future activities



• an opportunity to reflect on your approach and improve it

• Can make difference between knowledge transfer and knowledge exchange

• Without any evaluation activity becomes more about dissemination

• Nothing is perfect – what fits best?

When do we evaluate? • Ideally before, during and

after an activity

• In reality often done at the end - OK for single event but not for longer piece of research

• Includes looking at the quality of content, delivery and impact on participants


Evaluation Tools

• Surveys and questionnaires – paper or online

• Collecting comments – visitors’ book, post-it notes etc

• Audience response system

• Interviews

• Focus groups

• Creative activities



Some simple ideas

Example output from google form – attendance at today’s session


Arts & Humanities 9 14%

Medicine, Dentistry & Health

10 15%

Science 11 17%

Social Science 21 32%

Professional Services 8 12%

Other 7 11%


Comment feedback

Post it summary - Tramlines

Comments about:

• University

• Street acts

• Winter Gardens

• Tramlines

BUT some unusual comments:

• Needs more squid

• Keep rocking - Monkey man

• Baby don’t hurt me, don’t hurt me, no more

• Knives are sharp!


130 post-its completed in 1 day


Some creative ideas


http://www.youtube.com/watch?v=8RlE8hp4WdI

Films and video

Feedback video from masterclass 1


“As a participant, I think it did work and reinforced (in other ways) some of the key issues of that early workshop, so doing the reflection/evaluation in that format was a useful process for me too”.



Some more complicated ideas

Festival of the Mind

• Visitor numbers

• Visitor comments

• Website stats: Page visit, unique visitors, page views, pages viewed per visit, average visit duration, visitors using a mobile/tablet, non-mobile visitors, new or returning visitors and language settings

• Feedback from academics and partners involved


Feedback on social media • Twitter followers: 1200 for main FOTM

account with an additional 20 accounts related to projects

• Facebook 670 fans and 1200 comments and visits in the week of the festival

• Blogs – over twenty different blogs set up or featured FOTM in the week of the festival.


Where can I go for help?

• Public Engagement Team

• Digital Engagement Team – social media

• CiCS – audience response system training, technical help and creative media


Guidance

http://www.publicengagement.ac.uk/how/guides/evaluation/resources

http://www.rcuk.ac.uk/Publications/policy/Pages/Evaluation.aspx

http://www.publicengagement.ac.uk/how/guides/evaluation/examples


Suzanne Spicer

University of Manchester

Opportunities for engagement – what is coming up?

Platforms:

• Children’s Festival (June ‘14)

• Doc/Fest (June ‘14)

• Off the Shelf (October ‘14)

• Festival of the Mind (September ‘14)

For further information contact:

• Greg Oldfield, Head of Public Engagement

• Amy Carter, Public Engagement Manager

• Fran Marshall, Research and Evaluation Manager

• Addison Building Room A100

• [email protected]


Questions


To Discover And Understand.

public engagement with research masterclass 10: some ideas .../file/ideas_for... · costing your...

Documents