introduction to social performance measurement by peter scholten date 14 may 2012 (09:30 hr - 16:00...

Introduction to Social Performance Measurement by Peter Scholten 1

Date14 May 2012 (09:30 hr - 16:00 hr)

VenueIndia International CentreSeminar Hall 340 Max Mueller Marg New Delhi 10003

Delhi Workshop: Introduction to Social Performance Measurement


Agenda

• Introductions and Expectations

1. General introduction: backgrounds of SPM2. How to get started with SPM

– Objectives– Requirements to get started

3. Methods, tools, databases, resources


About Peter Scholten


Alice asked: “Cheshire cat…would you tell me, please, which way I ought to go from here?”

“That depends a good deal in where you want to get to. If you don’t know where you are going, then any road will take you there, said the cat”.

Social Performance Measurement


Introductions

• Your name & organization• What are your expectations for this

workshop?• What is your experience with Social

Performance Measurement?


General Introduction


Venture Philanthropy• To build stronger social purpose organizations by

providing them with both financial and non-financial support in order to increase their societal impact.

• Impact may be financial, social (incl. cultural, medical, etc.), environmental.

• key characteristics: high engagement, tailored financing, multi-year support, non-financial support, involvement of networks, organizational capacity-building and performance measurement


Charity: grant-

dependent

Charity: grants

and trading revenue

Social Enterprise: <75% trading revenue

Social Enterprise: break-

even

Social Enterpris

e: profitable, reinvest surplus

Socially-driven

Business: distribute

some profits

Business: Sustainable business practices

Business: allocate %

profit to charity

CSR ++

Business: mainstrea

m company

Organisations can create “blended” social and financial value

Key driver: Create social value

Key driver: Create

financial value

Source: Adapted by AVPN from Skoll Centre, EVPA, CAF Venturesome, Noaber Foundation

Impact Only

Venture Philanthropy

Finance FirstImpact First

Impact Investing

VP Spectrum


Social Purpose

Organisation

Venture Philanthropy multiplies the impact of financial capital through advisory services and high engagement

Increased SocialImpact

VenturePhilanthrop

yOrganisatio

n

Financial Capital

• Strategy• Marketing• Legal• Accounting• Networks• Coaching

Non-financial resources

Other Funders

Co-investment

Venture Philanthropy


Social Performance Measurement (SPM)

• Social Impact Assessment• Impact Measurement• Evaluation• Performance Measurement• Social Return

• There are so many names for similar activities: Need for Glossary


Development in SPM

• Trust me (Charity) < 1960’s

• Tell me (Philanthropy) 1960-2000

• Prove me (Venture Philanthropy) > 2000


The need for SPM

1. The management perspective

2. The market perspective

3. The government perspective

4. The public perspective

5. Critique


Management Perspective

1. Increasing social impact

2. Proof for success of a project

3. Improving performance

4. Accountability

5. Well-informed decision making by CEOs and boardmembers.


The Market Perspective

1. Efficient resource allocation

2. Creating benchmarks and standards

3. Attracting new resource providers

4. Blurring of borders between sectors (venture philanthropy, social venturing)


The Government Perspective

1. Clarifying tax exemption

2. Avoiding of scandals in philanthropic sector

3. Achievements instead of endowments (focus change)


The Public Perspective

1. Transparency

2. Communication tool

3. Publicity

4. Donor’s donations decisions


Comments on increased evaluation

1. Data obsession2. Trust based relationship between partners3. Numbers can create errors4. Comparable (apples-oranges and apples-

kangaroos)5. Externalities6. Focus on projects, outcomes and indicators

that are easy to measure


Apples and Oranges


How to start/continue


Theory of Change / Logic Model

• What is the issue you are trying to solve/prevent?• How big is it, what is the size?• How urgent is it? Wish or need?• What is your value added?

• If the Theory of Change is not clear, performance measurement is almost impossible


Be specific !

• “Increased Income”– 10 pesos per year– For one worker or the family– 5% increase (when inflation rate is 10%?)– In one year, or 5 years?– Disposable income for families?


Audience & Objectives

Audience Objective

Internal

External

Accounting

Learning


Timeframe & Perspective

timeframe perspective

Ex Post

Ex Ante

Investee

Investor


0

benefits

years cost

Short term or long term?


Trend

350

250

200 xx

150

100

50

2008 2009 2010 2011 2012 2013


350

250

200 xx

150

100 x

50x2008 2009 2010 2011 2012 2013

Trend


Frequency & research type

Frequency Research type

Due diligence

monitoring

Involve stakeholder

Desk research


Verification

• Will target group be asked about their experiences, expectations, opinions?– Can you claim an impact without asking the (main)

stakeholder?• Information from desk research?

– To what extend is research data applicable to your project/investment?


How & What

how

what

Qualitative

Quantative

Process

ISO, TQM BSC, ‘lives touched’

Impact

Story telling SROI, CBA


Mission Alignment

0

200

400

600

800

1000

1200

1971 1981 1991 1999

income

square acres

memberships


Planning SPM

• What do you need?• Who do you need?• When do you need it?

• Theory vs Practice


What is needed?

• Money:– Outsourcing (f.e. external audit)– Instruments, registration systems– Training (consultancy)

• Time– How much time is available? – Deadlines, busy, “it is extra”, (primairy process has

priority); – Patience vs. quick results


Who do you need?

• Internal: Colleagues• Internal support

– Management, IT, admin, etc.

• Who is in your team?• What knowledge/capabilities

• External:– Targetgroup(s)– Stakeholders– Support (consultant, university, etc.)


When do you deliver?

• Project planning– Gant chart / measurement plan

• What can you expect and when?• Who is expecting results and when?• Feasibility


Sign contracts and disburse capital

Y1 Y2 Y3 Y4

Preliminary review

Deal execution Portfolio controlling & reporting ExitDeal screening

Who uses the information of the framework?

How is the information of the framework used?

LGT VP Team Board LGT VP Team/ Board/Orgs Team + Board + Funders/Investors + Public

• Initial understanding of impact

• Decide if org will be presented to the board

Assess and report on achieved social impact (qualitative and quantitative)

Impact Model light

Impact Model

Investment memo (impact targets)

Due diligence Post-Investment Monitoring

Impact reports (internal + public)Deliverables

• Deeper understanding of impact

• Board approval of resources for local DD

• Board approval of engagement

• Define impact targets

Impact model framework contributes to decision making, management of portfolio and stakeholder reporting

Source: LGT Switzerland


Resistance or Support


…. reasons not to do performance measurement…..

• It is complex and difficult• It is expensive and time consuming• There are no incentives…• It is not standardized…• The methods are not scientifically approved…• We already do so much …• Is it my responsibility?


Disease of convexity “We create high value”


Methods, tools & databases


Where to start…?

rformance Measurement: Measurement:cial Impact As-Performance Measurement: 53.800.000Impact Assessment: 93.300.000Social Impact Assessment tool 32.000.000essment tool 32.000.000ent: 93.300.000


100’s of tools…

• Different background – American models, American models- European version, European

models, Other models• Different sectors

– Sustainability /, community-based / Development • Different purposes

– Results Oriented (quantitative); Process Oriented (qualitative)• Different knowledge levels

– Practitioner tools (many “home-made-tools”);Academic tools Consultancy tools

• Different networks– Academic, practitioners, consultants, for different sectors– many EU-networks through european subsidies


Examples

• Balanced Scorecard• Progress out of Poverty Index (PPI)• Social Return on Investment

– Cost Benefit Analysis, QALY’s• Participatory Impact Assessment

– Valuegame


Balanced Scorecard

Introduction to Social Performance Measurement by Peter Scholten

Source: One Foundation, Ireland

Attract and develop a diverse & talented Team of staff, volunteers and Board


Social Impact

Direct beneficiaries: MRCI provides valued & effective information, services & support to migrant workers & their families

Customer /Constituent

Internal Processes

Learning and Growth

Build evaluation capacity to drive performance over time


Ensure an effective work environment that promotes learning & participation


MRCI has sustainable and growing financial capacity to achieve its vision & makes effective use of resourcesMRCI has sustainable and growing financial capacity to achieve its vision & makes effective use of resources

Develop an effective referral & information sharing network


Advocacy targets: MRCI develops effective relationships with advocacy targets (government departments, unions & employers, community orgs, +)


• Balance the books

MRCI builds a strong reputation and influencing capacity


MRCI ensures internal cohesion through integrated approach to our work


MRCI actively manages growth –systems, governance & infrastructure


Provide credible evidence for advocacy positions from effective data gathering and analysis systems


DIRECT: Migrant workers & families can access their rights on 3 issues (access to services, workplace exploitation, participation)through MRCI case work & others


INDIRECT: The rights of migrant workers & their families on 3 key issue areas are advanced through the 3 integrated activities of MRCI (community work, DIC, advocacy)


• Meet key targets in Financial strategy

• Customer Satisfaction measured funding mix targets met • Quality of relationship with key advocacy targets

• No of satisfactory outcomes in issue areas

•Number of people served

• Specific policy achievements

• No of cases that have potential to be precedent-setting

•Quality participation in community work and policy activities by migrant workers and their families

•Meet key targets in

•communication strategy•Positive organisational culture

• Effective governance system in place

MRCI delivers excellent info case work, & community work

Funders: MRCI develops a mix of sustainable funding

lines

Financial

Number of data driven reports disseminated

Evaluation capacityNumber of opportunities for analysis,

learning taken up by range of stakeholders

Level of diversity within the Board, team and volunteers

Effective Specialist and local referral system in place



Social Impact

Direct beneficiaries: MRCI provides valued & effective information, services & support to migrant workers & their families

Customer /Constituent

Internal Processes

Learning and Growth





MRCI has sustainable and growing financial capacity to achieve its vision & makes effective use of resourcesMRCI has sustainable and growing financial capacity to achieve its vision & makes effective use of resources





• Balance the books













• Meet key targets in Financial strategy

• Customer Satisfaction measured funding mix targets met • Quality of relationship with key advocacy targets

• No of satisfactory outcomes in issue areas

•Number of people served

• Specific policy achievements

• No of cases that have potential to be precedent-setting

•Quality participation in community work and policy activities by migrant workers and their families

•Meet key targets in

•communication strategy•Positive organisational culture

• Effective governance system in place

MRCI delivers excellent info case work, & community work

Funders: MRCI develops a mix of sustainable funding

lines

Financial

Number of data driven reports disseminated

Evaluation capacityNumber of opportunities for analysis,

learning taken up by range of stakeholders

Level of diversity within the Board, team and volunteers

Effective Specialist and local referral system in place


Source: OneFoundation, Ireland

Perspective No Measures Q1 Goals Q2 Goals Q3 Goals

Social Impact 1

No. of satisfactory outcomes in issue areas (casework) system in place benchmark established

Social Impact 2 No of people served 90 per month 100 per month 120 per month

Social Impact 3

INDIRECT: Specific policy achievements

campaign issues identified on 2 out of

3 areas

Campaign plans for 3 key issue areas

developedPrioritise action

plans and set goals

Social Impact 4

INDIRECT: No of cases that are precedent setting

legal panel established

discussion paper re legal strategy

developed

Financial 5 Meet key targets in funding strategyState funding

application made secure AP fundingfollow up state

funding application

Financial 6 Balance the books Finance sub group

established

Customer 7 Customer satisfaction of service users

customer satisfaction trial

conducted

Customer 8 Funding mixFunding strategy

initiatedFunding strategy

finalised

Customer 9

Quality of Relationship with key advocacy targets

Identify key targets & enablers for 3 priority campaign

issues

establish contact with advocacy targets on 2

campaign issues

Internal Processes 10

Quality participation in community work / policy activities by migrant workers

appropriate evaluation systems

researched


Results - One Foundation

• It has been supplemented by an external evaluation (funded by co-investor) that captured qualitative elements of progress e.g. impact of policy programme at 2 year point

• There has been buy in from management team to the tool; but not used for Board reporting – not yet full alignment

• Results have largely been ahead of target – lives touched, policy influence, organisational development, EXCEPT fundraising/funding diversification

• Much focus on outputs• To support strategy implementation (process) in the funded

organisation rather than demonstration of the impact of any particular decision


Progress out of Poverty Index

• Estimates the likelihood that clients fall below the national poverty line;

• PPI is country specific and based on that country's best nationally representative income and expenditure household survey.

• The process starts with a nationally income and/or expenditure survey. These indicators are then tested and vetted with local MFIs and their representatives.


PPI - Process

• Staff visit clients to collect key information, such as family size, children attending school, housing, etc..

• Indicators are drawn from national household surveys (such as Mexico’s INEGI database or Pakistan’s Integrated Household Survey) or World Bank Living Standards Measurement Survey,

• This index serves as a baseline from which client progress is measured.

• In this way, individuals are ranked according to the applicable poverty line. Using this analysis, institutions can assess the poverty likelihood of clients by branch, by rural or urban setting and by client.


PPI Form


PPI - results• How effective are the MFI’s at alleviating poverty?• Is social outreach and outcome information available by country?• Are investment dollars reaching the poor and very poor?• At the portfolio level, how many clients are moving out of

poverty?• How long does it take for MFIs, on average, to attain for their

clients sustained movement above the poverty line?• What measures are taken to ensure that investments really

support the poor and poorest populations? Poverty assessment tools? Social audits? Other?

• developed by the Grameen Foundation, the Consultative Group to Assist the Poor (CGAP), and the Ford Foundation

• www.grameenfoundation.org

http://www.grameenfoundation.org/


Social Return On Investment

• Social value created by a capital investment in change• Quantifying all inputs and outcomes into monetairy

values• Ratio between investments and benefits• Enable to aggregate results and portfolios• Focus on:

– Stakeholder involvement– Valuation in order to compare investments

• Money as a calculation unit; not real money

Source: www.thesroinetwork.org

http://www.thesroinetwork.org/


Reporting

Analysing

Social Return on Investment

1. Involve stakeholders

2. Understand what changes

3. Value the things that matter

4. Only include what is material

5. Do not over claim

6. Be transparent

7. Verify the result

Principles of SROI


Participatory Impact Assessment


ValueGame

• Perceived value for primary stakeholder group(s) – not institutions– For institutions mainly cost-savings or re-allocations

• Focus group- and/or online research to measure the importance of the change for specific stakeholder in specific situation

• Ranking and rating of outcomes using images• Beyond satisfaction research: you can be

satisfied with something that has low value


Value

Price

Costs

Value - Price - Costs

“Value is in the eye of the stakeholder”


ValueGame


Databases

• IRIS-GIIN• WikiVOIS• Worldbank


Source: global impact investing network, http://iris.thegiin.org/iris-standards

http://iris.thegiin.org/iris-standards




http://iresearch.worldbank.org/lsms/lsmssurveyFinder.htm


Online tools

• Tools of change• Impactmap• Social Evaluator• ValueGame


www.toolsofchange.com

http://www.toolsofchange.com/


www.impactmap.org

http://www.impactmap.org/


www.socialevaluator.eu

http://www.socialevaluator.eu/


www.valuegame.org

http://www.valuegame.org/


The best tool/method/database…

• There is not something like ‘the best tool’…• It all depends on your scope, objectives and

resources;• Most methods use similar framework (change,

stakeholders, impact, etc.); more commonalities than differences;

• How to make your choice?– EVPA Manual for SPM (due February 2013)– Integrate AVPN-experiences &knowledge


EVPA and its Knowledge Centre are supported by:

Concrete Plans for the IMI 3

Webinar Series April – July 2012

Case Study DevelopmentApril – August 2012

1st Draft Manual & WorkshopOctober 2012

“Version 1.0” ManualFebruary 2013


Contact information

• Peter Scholten• PO Box 59695• 1040 LD Amsterdam• The Netherlands• Email: [email protected]• Mobile: +31-6-17430741


Become a Member

Become a Sponsor

Support AVPN with a donation

Register to receive updates

Attend our events

website: www.avpn.asia

email: [email protected]

AVPN is a registered charity in Singapore (UEN:

201016116M)

Engage with Us

introduction to social performance measurement by peter scholten date 14 may 2012 (09:30 hr - 16:00...

Documents