Cultured monitoring and evaluation – getting it right

Steven Marwick

Evaluation Support Scotland

Who we are

• Steven Marwick, Evaluation Support Scotland

• Fiona Campbell, Voluntary Arts Scotland

• Fiona Rogan, Scottish Community Drama Association (and serial volunteer!)

Evaluation Support Scotland

We provide specialist support to voluntary organisations and funders to enable them to evaluate and learn and so provide better services for communities.

• Website and resources• Workshops• Tailored support• Partnership with others

• What is evaluation?

• What stops it happening?

• How do we get it right?

What do you think?

And what do you know?!

Starting point

1= not really 4 = yes definitely

1. Evaluation is important to my organisation.

2. We know what difference we are trying to make.

3. We collect information about what difference we are making.

4. We take time to analyse and report that information.

What is evaluation?

Evaluation is a process of asking questions

gathering evidence

analysing it

being prepared to act on the results.

It’s easy!!

Serious stuff • Lack of skills, capacity and time to evaluate -

voluntary orgs, funders, policy makers.• Evaluation seen as someone else’s job.• Outcomes can be tough to express

and to measure.• Some organisations don’t collect • the right information.• Few take time to analyse it.• We are confused by the language.• The products of evaluation

often not used for learning.

M&E

• Collecting output info

• Collecting outcome info

• Analysing output info

• Analysing outcome info

And LEARNING!

Outcomes!

Outcomes are the changes or difference you want to make through your services or

activitiesSkillsKnowledgeBehaviourFeelingsCondition (health, environment)

The science bit• OutCOMEs are the changes and

differences come out of your activities.

• OutPUTs are the activities or services you put on for your users.

• Inputs are the resources you need to do the work: money, people, time, premises, equipment.

Input output outcome impact

Singalong event

What difference or changes should I hope to make?

• To learn more about music• To improve your ability to sing• To meet new people• To have a good time

These are outcomes!

I nput Output Short term outcomes

Medium outcomes

Longer term outcomes

Vol’teers Music Conductor

Nos of people who sang A concert?

Learn more about music

Improved singing ability

Meet new people

Have a good time

Continued participation in music

Increased confidence

Improved social networks

More volunteering

More creative Scotland

Less isolation, more engage’mt

Collective confidence

Whose outcomes?

For example – Bill joins a running club:

• To get better at running? (excellence)• To get fitter? (health)• To reduce isolation? (inclusion)• To stop me getting into trouble? (crime)

Participants’ views matter.

Where are the outcomes?

• From participants (and audience?)

• Observation

• Third parties

• Research / stats

Making evaluation part of the performance

• What do we collect already?

• Is this evidence of outcomes or outputs?

• What new information do we need?

• How can we build information gathering into activities?

• How can we make it creative?

• Are we able to learn?

Fiona Campbell, Voluntary Arts Scotland

Fiona Rogan, Scottish Community Drama Association

In conclusion …

• Evaluation matters!

• It’s about learning

• It should be as creative and participative as our practice

• It’s not just numbers

Review

Never B flat

Never B sharp

Always B natural

Questions for us

Questions for you

1.What evaluation does your do and what do you use evaluation for?

2.Do you encourage organisations you fund to use evaluation for learning and development?

3.Are the evaluation methods and systems we have in place currently fit for purpose. If not, what would improve things?