Outcomes- Based Evaluation: what we can learn from museums?

Suzanne Spicersuzanne.spicer@manchester.ac.uk

Today’s programme

1. Background to outcome-based evaluation in museums

2. What can we learn from museums?

3. Using these lessons to develop an evaluation plan

The cultural change in museums

• ‘A Common Wealth: Museums and Learning in the UK’, A report to the Department of National Heritage, Jan 1997

• “Education, education, education”

• Change from “Education” to “Learning”

• Desire to become 21st century learning organisations• Launch of quality framework by MLA (Museums, Libraries & Archives Council)

Inspiring Learning for All A good practice framework based on 4 key principles

www.inspiringlearningforall.gov.uk

Evidence of the impact on learners• External drivers eg funding, public policy, regulatory

• Practical planning benefits

• Support continual improvement

• Demonstrate what can contribute to learning agenda

“turning fuzzy anecdotal evidence into credible data for advocacy and service development”

Generic Learning Outcomes: measure impact on learning

Generic Social Outcomes measure impact on social & community life

Health and well-being

Stronger and safer

communities

Strengthening public life

What do the GLO/GSOs offer museums?

• Provide a common conceptual framework and a systematic approach• Provide agreed categories for classification• Enable move from ‘anecdote’ to ‘evidence’• Is used to describe breadth and depth of outcomes• Links quantitative and qualitative research• Useful planning tool • Generated confidence in undertaking evaluation

Evaluation planning:Writing an evaluation plan

• Overarching aim (what do you want to achieve)

• Objectives (SMART)

• Evaluation questions you want to answer (outcomes/indicators)

• Methodology (techniques & the mosaic approach)

• Evidence (information you will collect)

• Analysis (coding and grouping your information)

• Reporting (lessons learned)

COLLECTING COMMENTS

OBSERVATION

Accompanied visitsBehavioural mapping

TrackingRecording (audio/video)

EVIDENCE-BASED DATA

DiariesPictures, logs

Outputs

TASK-ORIENTED

TalkingCreative activity

Drama

INTERVIEWS

Focus groupsIndividual

Personal Meaning MappingStructured/unstructured

QUESTIONNAIRES

Mail, e-mail, phone, webSelf-completion, face-to-face

Answer phone, letters, faxPost-Its

Comment books, boards, boxesStaff

Graffiti Wall

DATA COLLECTION

Ask for comments creatively; on a washing line or post-its on walls

Voluntary Arts Wales evaluation guidance publication: Tear Up Your Tick Boxes

Encourage doodles

What did you enjoy most today?

Why did you enjoy it so much?

Mosaic approach - triangulationSpaces to Play: more listening to young children using the mosaic approach, Alison Clark & Peter Moss (2005), London, National Children’s Bureau

The mosaic approach:

• Uses different methods to build a picture of the project

• Combines quantitative and qualitative data

• Includes visual methodologies

• Avoids reliance on the written or spoken word

Event

Staff graffiti wall

Adult questionnaire

Child’s drawings

Useful linksESRC Communications Toolkithttp://www.esrc.ac.uk/ESRCInfoCentre/CTK/default.aspx

RCUK Practical Guide to dialogue with the publichttp://www.rcuk.ac.uk/cmsweb/downloads/rcuk/scisoc/dialogue.pdf

RCUK Evaluation Guidehttp://www.rcuk.ac.uk/cmsweb/downloads/rcuk/publications/evaluationguide.pdf

AHRC Guide to Self-Evaluationhttp://www.ahrc.ac.uk/FundedResearch/Documents/Understanding%20Your%20Project.pdf

EPSRC Good Practice Guidehttp://www.epsrc.ac.uk/SiteCollectionDocuments/form-notes/ppe-goodpracticeguide.pdf

RCMG, University of Leicesterhttp://www.le.ac.uk/ms/research/rcmg.html

Inspiring Learning for Allwww.inspiringlearningforall.gov.uk