SOCIAL RETURN ON INVESTMENT (SROI)

Creative Communities Consortium

1

Kerry Traynor

A STEP BY STEP GUIDE

7 Principles

6 Stages of SROI Process 1) Establishing Scope & Identifying Key Stakeholders

2) Mapping Outcomes

3) Evidencing Outcomes & Giving Them a Value

4) Establishing Impact a. Consulting Stakeholders

b. Developing Monitoring Tools

5) Calculating the SROI

6) Reporting, Using & Embedding

(A Guide to Social Return on Investment, Cabinet Office, 2009)

2

Kerry Traynor

HOW THE PROCESS WILL WORK

Monthly meetings

Focus on each stage of SROI

Workshop approach: presentation & discussion

Completion of SROI worksheet

You - in consultation with your stakeholders

Homework! Finishing off and checking

SROI analysis completed within 6 months

Use shared monitoring forms and keep data safe

3

Kerry Traynor

OUR RESPONSIBILITIES

Me Guide you through the process

Provide accurate and honest information

Source benchmarking data

SROI calculations

Develop final report

You Attend workshops or arrange catch-up

Think carefully, be honest and give as much detail as you can

Consult your stakeholders

Do your homework!

4

Kerry Traynor

7 PRINCIPLES OF SROI

Involve stakeholders

stages 2, 3 and 4 require stakeholder consultation

stakeholder-informed not stakeholder-led

Understand what changes

Value the things that matter

Only include what is material

Do not over-claim

Be transparent

Verify the result

5

Kerry Traynor

STAGE 1.1 – ESTABLISHING SCOPE

1. Purpose

2. Audience

3. Background

4. Resources

5. Who will carry out the work?

6. The range of activities on which you will focus

7. The period of time to be considered

8. Type of analysis

6

Kerry Traynor

STAGE 1.1 – 1. PURPOSE

1. What is the purpose of this SROI analysis?

2. Why do you want to begin this process now?

3. Are there specific motivations driving the work, such as

strategic planning or funding requirements?

The purpose of this SROI analysis is to explore the social value

being created, both individually and collectively, by an emerging

network of creative and social organisations. A key aim of the

analysis is to explore the extent to which the network is

contributing to the proposed social change prioritised by the local

authority’s new strategic regeneration plan, A Vision for Sefton.

7

Kerry Traynor

STAGE 1.1 – 2. AUDIENCE

1. Who is this analysis for?

2. This should cover an initial assessment of how you will communicate with your audiences

This analysis is aimed at stakeholders interested in funding or otherwise supporting organisations within the network. It is also aimed at communicating the impact of the network to a broader audience including potential participants, partner organisations and other local organisations in the creative and social sectors.

Some stakeholders will be engaged in the analysis through consultation or through other means. On completion of the analysis, a range of key stakeholders will be invited to a launch event for the final report, and invited to comment. The final report will also be made available online and parts of it will be used to inform a variety of communications and funding bids.

8

Kerry Traynor

STAGE 1.1 – 3. BACKGROUND

1. Consider the aims and objectives of your organisation and how

it is trying to make a difference.

2. If you are focusing on specific activities you will need to

understand the objectives of those activities.

3. It is important that you have a clear understanding of what

your organisation does and what it hopes to achieve by its

activities.

The Creative Communities Consortium aims to … by …

[Your organisation] aims to … by …

9

Kerry Traynor

STAGE 1.1 – 4. RESOURCES

1. What resources, such as staff time or money, will be required?

2. Are these available?

Workshops: 8 hours (8 x 1hr workshops Jan – July 2013)

Homework: 8-16 hours (1-2 hrs per month)

Consultation: 4-6 hours

TOTAL: 20-30 hours

10

Kerry Traynor

STAGE 1.1 – 5. WHO WILL CARRY OUT THE

WORK?

1. Can you undertake the SROI analysis internally, or will you need to bring in external help?

2. Make sure you have the right mix of skills and support from the start.

3. Generally, you will need skills or experience in finance, accounting, evaluation and involving stakeholders.

An external associate will lead the analysis, guiding organisation/ project representatives through the process step-by-step. She has experience in the areas noted above, as well as in research and management in the creative and social sectors, and teaching in higher education in relevant subjects. External funding has been secured to cover costs. The network/associate also have access to a range of forums and support organisations providing information and advice on the SROI process.

11

Kerry Traynor

STAGE 1.1 – 6. RANGE OF ACTIVITIES

1. Will you be analysing all the activities of your organisation, or just specific ones?

2. You might want to separate the activities related to a particular source of funding, or those that are a priority for you.

3. Keep your scope small if this is your first SROI.

4. Clearly describe what you intend to measure. For example, if the activity is “our work with young people” this may cover several different projects or services. You may actually mean something more specific, like “mentoring support provided to young people”.

The aim is to analyse all of the KEY activities of member organisations. However it is recognised that this may not be feasible within the timescales and with the resources available within organisations. It is therefore recommended that members prioritise the projects/services you wish to include in the analysis and work through them one by one within the time you have available.

NB. Not including one or more projects will not necessarily have a negative impact on your final SROI calculation, since both the relevant inputs and outputs will be omitted from the final calculations.

12

Kerry Traynor

STAGE 1.1 – 7. PERIOD OF TIME

1. SROI is often annual

2. Often corresponds with financial accounting timescales

3. Can be specific to a project timescale

The analysis will cover ONE YEAR, corresponding with the

majority of members’ financial accounting timescales.

This means including all activities/projects/services that will take

place within that year, even if they do not run continuously

throughout the year.

13

Kerry Traynor

STAGE 1.1 – 8. TYPE OF ANALYSIS

Type of analysis Evaluative

Conducted retrospectively

Based on actual outcomes that have already taken place

Can be challenging if correct data has not been collected

Forecast Predicts how much social value will be created if activities meet intended outcomes

Especially useful in planning stages

Help to show how investment can maximise impact

Useful for identifying what should be measured once project is up and running

Ensures that right data is collected to enable an evaluative SROI in the future

We will be undertaking a FORECAST analysis.

On completion of the first year, an EVALUATIVE analysis will be undertaken, comparing the forecast with actual outcomes.

14

Kerry Traynor

STAGE 1.1 – ADJUSTING THE SCOPE

Adjusting your scope in response to new information is good practice and is not unusual

You may wish to review your scope after considering the numbers and types of stakeholders you need to involve

This will determine the resource required and it may mean you need to start with fewer activities

We will revisit our scope at the relevant stage.

Please keep your scope in mind during the initial stages and make a note if you think you need to review your scope.

15

Kerry Traynor

STAGE 1.2 – IDENTIFYING STAKEHOLDERS

Stakeholders are identified as people, groups or organisations

that experience change as a result of your activities

Don’t hold back – identifying a stakeholder group here does

NOT mean you will have to engage or consult them all!

Now decide which stakeholders are most relevant to your

activity and should be included in your analysis

Make a note explaining your decisions for including or excluding

each stakeholder – we will need these at a later date

Now transfer the names of the stakeholder groups you have

chosen to include onto the SROI Impact Map – Stage 1

16

Kerry Traynor

STAGE 1.2 – IDENTIFYING CHANGES

Describe the key change(s) that each stakeholder group

experience

Changes may be positive or negative

Changes may be intentional or unintentional

Be descriptive, but succinct!

17

Kerry Traynor

STAGE 2 – MAPPING OUTCOMES

2.1 Beginning your Impact Map

2.2 Identifying Inputs

2.3 Valuing Inputs

2.4 Clarifying outputs

2.5 Describing outcomes

Remember our themes:

Improving Health & Wellbeing

Improving Skills, Employability

Improving Organisations

Improving Physical Spaces

18

Kerry Traynor

STAGE 2.1 – BEGIN YOUR IMPACT MAP

A different Impact Map for each key activity/service/project

Enter the following information at the top of the form:

Name of Organisation

Organisation Objectives

Activity

Activity Objective

Contract/Funding/Dept

Purpose of Analysis (this will always be Forecast)

Your Name

Date of Completion

19

Kerry Traynor

STAGE 2.2 – IDENTIFYING INPUTS

Investment refers to the financial value of the inputs

Identify the contributions of stakeholders

ESSENTIAL to the delivery of the activity

Inputs are used up in the course of the activity

Mainly money, time or in-kind goods and services

Funders or customers might invest money

Volunteers or partners might invest time

Beware of double-counting inputs – eg. if you have a surplus

Do NOT include time contributed by beneficiaries

20

Kerry Traynor

STAGE 2.3 – VALUING INPUTS

Create financial values for all inputs

Regardless of whether any money changes hands

Ensures transparency about the full cost of your activity

For volunteers – use a value equivalent to average hourly rate

for the type of work they are doing

For volunteers – can also include an allocation of overheads as

you might with an employed person eg. NI, desk space, IT etc.

Volunteering England has some useful information

21

Kerry Traynor

STAGE 2.4 – CLARIFYING OUTPUTS

A quantitative summary of the work undertaken

50 people trained

25 children involved

6 artworks created

5 businesses created

May be several different outputs for each activity or each

stakeholder group

Outputs can be repeated for different stakeholder groups

22

Kerry Traynor

STAGE 2.5 – DESCRIBING OUTCOMES

An outcome is what happens as a result of the output

50 people trained = 10 trainees get jobs

25 children involved = 20 children improve confidence and self esteem

6 artworks created = 6 playgrounds improved

5 businesses created = jobs created

Relate outcomes to the stakeholder group

Broad stakeholder groups might have broad outcomes

Eg. A govt funded project for refugees might list “better integration of refugees” as an outcome for the state

Use your judgement

23

Kerry Traynor

STAGE 2.5 – DESCRIBING OUTCOMES

An outcome is what happens as a result of the output

50 people trained = 10 trainees get jobs

25 children involved = 20 children improve confidence and self esteem

6 artworks created = 6 playgrounds improved

5 businesses created = jobs created

Relate outcomes to the stakeholder group

Broad stakeholder groups might have broad outcomes

Eg. A govt funded project for refugees might list “better integration of refugees” as an outcome for the state

Use your judgement

24

Kerry Traynor

STAGE 2 – REVIEWING YOUR IMPACT MAP

This chain of events is often described as a theory of change

Make sure you have only included material outcomes

Check that you aren’t missing anything significant or including

something that is not relevant

Decide what you will finally include before you move on to the

next stage of measurement

If you decide to exclude any outcomes make a note of this and

your reasons – we will need this at a later stage

25

Kerry Traynor

STAGE 3 – EVIDENCING OUTCOMES AND GIVING

THEM A VALUE

Recap previous session Remember outcomes should include numbers

3.1 Developing outcome indicators

3.2 Collecting outcomes data

3.3 Establishing how long outcomes last

3.4 Putting a value on outcome

Remember our themes: Improving Health & Wellbeing

Improving Skills, Employability

Improving Organisations

Improving Physical Spaces

26

Kerry Traynor

TOOLS – HEALTH & WELLBEING

Improving Health & Wellbeing

Rosenberg Self Esteem Scale

Warwick Edinburgh Mental WellBeing

Scale (WEMWBS)

Outcomes Star Wellbeing

Outcomes Star Mental Health Recovery

Clinical Outcomes in Routine Evaluation

(CORE) - 34-item questionnaire measuring

“a pan-theoretical core of clients' global

distress”, including subjective well-being,

commonly experienced problems or

symptoms, and life/social functioning”

27

Kerry Traynor

TOOLS – CHILDREN’S WELLBEING

Leuven Scale of Child Involvement – measures engagement in activity

UNICEF’s 2008 report Child poverty in perspective: An overview of child well-being in rich countries presents a new index of childhood well-being, constructed around six domains, one of which was subjective well-being (defined in terms of self-reported satisfaction with health, school and life overall).

The TellUs survey, developed and conducted by OfSted, is an extensive (13 page) self-report survey based around the five core dimensions of the ECM framework. It includes a question on happiness.

New Philanthropy Capital – Feelings Count – developed specifically for use in the charity sector for use with 11–16-year-olds. 41 questions incorporating 5 existing models. Seven core aspects of subjective well-being incl. self-esteem, resilience, emotional well-being, peer relationships, family relationships, satisfaction with school environment, and satisfaction with local community environment.

NEF/Action for Children developed a sample questionnaire for primary school aged children using a ladder to rate overall life satisfaction and smiley faces to rate 9 questions about emotional wellbeing, relationships and physical indicators of stress.

The Warwick Edinburgh Mental Well-being Scale (WEMWBS) is a 14-item instrument that was developed for NHS Health Scotland and (from 2008) is included in the core module of the annual Scottish Health Survey – now validated for use with 13-15-year-olds.

28

Kerry Traynor

TOOLS – SKILLS AND EMPLOYABILITY

Improving Skills & Employability

Outcomes Star Work – Triangle Consulting

Distance Travelled Guide – developed by ESF

Formula One – developed for Neighbourhood Learning Centres

The Personal Power Pack - developed by Gloucestershire Council Adult Education

The SOUL (Soft Outcomes Universal Learning) Project – developed by the Norfolk

voluntary and community sector

29

Kerry Traynor

TOOLS – IMPROVING ORGANISATIONS

Jobs created/safeguarded

Financial indicators -Turnover, ratio of public/private revenue, surplus, stability

PQASSO spider diagram

1. Planning

2. Governance

3. Leadership and management

4. User-centred service

5. Managing people

6. Learning and development

7. Managing money

8. Managing resources

9. Communications and promotion

10. Working with others

11. Monitoring and evaluation

12. Results

30

Kerry Traynor

STAGE 3.1 – DEVELOPING OUTCOME INDICATORS

An indicator is a way of finding out whether and how much change has happened

Create at least ONE indicator for each outcome

More indicators will strengthen (triangulate) your data

Balance between subjective/perceptions and objective/verifiable

Subjective and objective can both be self reported, but third party data would strengthen a case eg. For health improvements, subjective data might be participant perceptions of improvements to health or wellbeing. Objective data might be number of medical appointments/medication reduced.

Check they are measurable and within our means

Remember our themes: Improving Health & Wellbeing

Improving Skills, Employability

Improving Organisations

Improving Physical Spaces

31

Kerry Traynor

STAGE 3.2 – COLLECTING OUTCOMES DATA

Identify WHERE you can collect data on outcomes from

Maybe already available from internal or external sources Membership organisations, government departments, market research firms,

consulting companies, partner organisations etc.

Published research from universities, government departments or research organisations etc.

Maybe you need to collect new data from stakeholders Beneficiaries, project staff, partners etc.

Identify HOW you will collect the data

Bespoke monitoring forms

121 sessions / Interviews / focus groups

Questionnaires (face-to-face, telephone, post, email, internet)

From partner/third party agencies (data protection issues…)

32

Kerry Traynor

STAGE 3.2 – COLLECTING OUTCOMES DATA

WHO will you ask?

Total population or representative sample? What proportion?

Random, purposive, probabilistic or convenience

Use your judgement – choose a sample you feel is defensible and within budget

WHO will collect it?

Do you have (can you make) time?

Think about bias / power relationships - can you swap with colleagues?

Is there anybody external?

WHEN will you collect it?

Plan ahead

Start, middle and end

Every 2 weeks / 6 months / annually

Post-project tracking?

During existing contact or separately?

33

Kerry Traynor

STAGE 3.3 – ESTABLISHING HOW LONG

OUTCOMES LAST

Some outcomes last longer than others – maybe 1-5 years

Some depend on continuing intervention, some do not

If you believe the outcome will last beyond the activity, then it

will continue to generate value

Timescale usually number of years – the benefit period

Estimate the duration of each of your outcomes

Ideally you would ASK your beneficiaries

If this is not possible you can use other research to support

your estimate

Do not overstate your case and lose credibility

34

Kerry Traynor

STAGE 3.4 – PUTTING A VALUE ON THE OUTCOME

Create financial values for all outcomes

Monetisation of the outcome – assigning proxies

Proxies = approximations

All values are subjective – markets decide

Widely used in environmental and health economics – less so

in the creative and social sectors

35

Kerry Traynor

STAGE 3.4 – PUTTING A VALUE ON THE OUTCOME

Health & Wellbeing

Cost saving for NHS of GP clinic appointments

Cost saving for NHS of counselling services

Improving Skills & Employability - moving into employment creates:

Increased income to individual - net of tax

Reduced costs to government - benefits

Improving Organisations

More efficient organisations reduce need for public subsidy

New jobs create increased income for the government – employment taxes

Improving Physical Spaces

Increased land use

Reduced vandalism

36

Kerry Traynor

STAGE 3.4 – PUTTING A VALUE ON THE OUTCOME

Proxies that are more challenging..

Contingent valuation – asking people directly how they

value things

Revealed preference – inferring valuations from the prices

of market-related goods

Travel cost methods – translates inconvenience into money

eg. costs of someone travelling to access a service

Average household spending – on categories such as

leisure, health or home improvement – to see how much

people value these types of activity (see Family Spending

Survey)

37

Kerry Traynor

STAGE 4 – ESTABLISHING IMPACT

4.1 Deadweight and Displacement

4.2 Attribution

4.3 Drop-Off

4.4 Calculating your Impact

38

Kerry Traynor

STAGE 4.1 – DEADWEIGHT

DEADWEIGHT is a measure of the amount of outcome that would

have happened even IF THE ACTIVITY HAD NOT TAKEN PLACE.

Eg. a regeneration programme finds that there had been a 7%

increase in economic activity since the programme began. However

the national economy grew by 5% during that time. So you would

need to consider how much of the 7% was due to your intervention.

This will always be an estimate – don’t worry about “proving” your

estimate.

We may be able to access national datasets such as British Social

Attitudes Survey to get an idea of nationwide trends for comparison.

39

Kerry Traynor

STAGE 4.1 – DISPLACEMENT

DISPLACEMENT is an assessment of HOW MUCH OF YOUR

OUTCOME DISPLACED ANOTHER OUTCOME.

Does not apply in every SROI but must be aware.

Eg. crime being moved from one area to another.

Eg. an employed person moving from one job to another – as

they are displacing a potential outcome of moving somebody

out of unemployment and into employment.

40

Kerry Traynor

STAGE 4.2 – ATTRIBUTION

ATTRIBUTION is an assessment of HOW MUCH OF THE

OUTCOME WAS CAUSED BY THE CONTRIBUTION OF OTHER

PROJECTS OR SERVICES.

Eg. a new cycling scheme finds a significant decrease in carbon

emissions. However a congestion charge was also introduced

and an environmental awareness programme began, so you

have to ATTRIBUTE some of the outcome to their contribution.

41

Kerry Traynor

STAGE 4.3 – DROP-OFF

Remember in section 3 we considered how long we think each outcome will last…

DROP-OFF is an assessment of the REDUCTION IN THE INFLUENCE OF YOUR INTERVENTION IN A LONG TERM OUTCOME.

Only complete this for outcomes that last LONGER than ONE YEAR.

Eg. Your intervention is a training programme which helps unemployed people return to work. In the first year you could justify that your intervention played a very large part in helping them get a job, However if they are still employed after 2 or 3 years then that may be due more to their own commitment, development of on-the-job skills and networking.

You can use a standard approach eg. 50% drop in Y2, 20% in Y2, 20% in Y3, 10% in Y4.

Eg. an outcome of 100% that lasts for 3 years and drops off by 10% per year would be 100% in Y1, 90% in Y2 and 81% in Y3.

42

Kerry Traynor

4.4 CALCULATING YOUR IMPACT & SROI

43

Kerry Traynor

NEXT STEPS

I will type up your Impact Maps to build in your work from today

I will email these to you ASAP with some suggestions for development

Before the next session please consider these comments and develop your map further if necessary

Please also give some thought to Stakeholder Consultation – we will plan this together in the next session but please think about which of your stakeholders:

Would be thoughtful and give a range of views – unexpected feedback is good!

Would be willing and able to comment on your analysis

Would be available during the time you have available

Think about past participants rather than current ones if possible

Consider asking funders, partners and non-users for their views

Remember you don’t have to meet them in person – an email/phone conversation is fine

Try to get a representative sample eg. cross age/gender/type

You don’t have to use/show the Impact Map if you don’t want to – think of other (creative!) ways of presenting and discussing the contents

44

Kerry Traynor