bep quarterly meeting: an emerging framework for proving & improving effectiveness

bond.org.uk

BEP Quarterly meeting: an emerging framework for proving & improving effectiveness January 2011

bond.org.uk 2

Issues to cover

• The BEP Story so far….

• The Effectiveness Self-Assessment

• The Prove and Improve It! Framework• Approach & purpose

• Building blocks

• Details

• Way forward

bond.org.uk 3

Story of BEP so far….

The best way of understanding how the Bond Effectiveness Programme has evolved is by looking at the different diagrams that have been used to represent an effectiveness framework…

Leadership and management

Governance and strategy

Financial management

People management Communications

Possible thematic areas:

MicrofinanceCapacity building

Development educationService delivery

Advocacy/campaigningPolicy research

Humanitarian reliefRelationships and alliances

Mission specific levers of effectiveness

Principles of development effectiveness

• Accountability to communities

Approaches to demonstrating your effectiveness

Learning & knowledge management

• Demonstrating & managing by results

• Respect for diversity & human rights

User / beneficiary focus

• Continually learning & improving

• Financial & political independence • Local ownership & partnership

Common levers of organisational effectiveness

Tools and approaches for measuring effectiveness of different thematic activities

Types of evidence for demonstrating change in specific areas of work

One World Trust - Comprehensive effectiveness review

2. Organisational Effectiveness Levers/DriversEg. ● Leadership & strategy Governance ● Financial management

● People management● Marketing & communications ● Continuous improvement and

learning● Relationships ● User/beneficiary focus

3. Sector specificLevers

e.g.•Microfinance•WASH etc (Refer to OECD Dac keywords etc)

4. Process Leverse.g. •Gender•Advocacy•Service delivery•Partnerships•Capacity development(Refer to 6 GRI indicators etc)

5. Measuring effectiveness● Tools ● Types of evidence

6. Using evidence for decision making &

continuous improvement

7. Communicating evidence of effectiveness to

stakeholders

Org

anisa

tiona

l Effe

ctive

ness

Deve

lopm

ent E

ffecti

vene

ssPr

ogra

mm

e Eff

ectiv

enes

s

1. Foundation principles for development effectivenessEg. ● Continuous learning ● Demonstrating & managing by results ● Respect for diversity/human rights● Accountability to stakeholders ● Financial & political independence ● Local ownership & participation

Prove and Improve It! Framework

Effectiveness Self-assessment and

online portal

Framework working group – revised effectiveness framework

An effectiveness self-assessment &

online portal on NGO

Effectiveness

(Illustrative) Drivers of program and organisational effectivenessLearning &

improvement HR capacity Partnership & participation Transparency Managing for

effectiveness Leadership & governance

Question 1 Question 1 Question 1 Question 1 Question 1 Question 1



Online portal on NGO Effectiveness

Learning & improvement HR capacity Partnership &

participation Transparency Demonstrating effectiveness

Leadership & governance

Tools, approaches, training, other initiatives






Step 1. Conduct effectiveness

self-assessment online

Step 2. Reveal strengths and weaknesses &

benchmark performance with

peers

Step 3. Go to online portal to identify how to

plug gaps

HR capacity

Transparency

Learning & improvement

Leadership & governance

Managing for effectiveness

Participation & partnership

1

2

3

4

5

NGO X

Sector average

Step 4. agree action plan,

ongoing support from peers

Establish peer groups to provide support in implementing action plans

bond.org.uk 6

Effectiveness self-assessment – moving forward

• Will be working with a consultant to develop the effectiveness self-assessment and another to design the online platform and portal

• Will be consulting the membership on this tool separately• Draft of self-assessment complete by July, with piloting

between July - Oct• Final version completed by Nov

bond.org.uk

Capacity buildingSouthern advo-

cacy

Northern advo-cacy

Service deliv

ery

Hum

anita

rian

resp

onse Ways of working

Thematic Areas Education

Rights

Women’s

empowermentLivelihoodsWater &

sanitation

Hea

lth

Pro

ve &

Impr

ove

It!

Fram

ewor

k

bond.org.uk 8

Walking you through the Prove & Improve It! Framework


1. Approach & purpose

2. The building blocks

3. Theories of change, indicators & methods

bond.org.uk 9

PIF unpacked: approach & purpose

• A tool grounded in the distinctive contributions that NGOs make to international development that will enable organizations to assess, manage and report their effectiveness confidently and consistently

• The framework will support organizations, depending on their ‘way(s) of working’ and thematic areas of work, in identifying:

• What to assess – common output, outcome and impact areas and indicators

• How to assess – common methodologies and approaches

• What to report – common understanding of what we should be communicating to different stakeholders and how

• The framework will provide the platform for the NGO sector to come together to agree on how it demonstrates effectiveness to itself and others

bond.org.uk 10






bond.org.uk 11

PIF unpacked: the building blocks Why use ways of working as the entry point to effectiveness?

1. Places NGOs distinctive contribution to international development at the heart of the

framework

2. A NGO needs to understand and be monitoring its contribution in order to be able to show

longer term impact

3. Provides a wide platform for NGOs to come together and identify common solutions to

assessing and demonstrating effectiveness

Ways of working are: “the distinctive contributions that NGOs make to international development; our added value”

Why have thematic areas?• Ways of working are a means to an end; they are pathways to supporting sustainable

change. The framework needs to link our approaches to change with whether or not we

are actually making a difference to the lives of poor and marginalised people

Thematic areas are: “the core areas in which NGOs seek to bring about change; how we categorize our results”

bond.org.uk 12






bond.org.uk 13

PIF unpacked: theories of change, indicators & assessment methods What will sit inside the ways of working and thematic areas?

Common

outcome

areas Common

indicatorsCommon approaches to measurements

Agreed ways of

communicating change

bond.org.uk 14

For each way of working:

1. Map out a general theory of change; broadly, what are the key areas in which we seek to bring about change and what are the links /relationships between them

2. For each step in the process of change answer three common questions:

I. What to assess? What are the best indicators to track progress? Quantitative / qualitative?

II. How to asses? What are the best methods for data collection and assessment?

III. What to communicate, to whom and how? What information should we be reporting throuhg

what methods?

For each thematic area:

• Define and agree top level impact and outcome indicators

• These would need to be adjusted to different ways of working eg capacity building for water and sanitation or direct service delivery in health

PIF unpacked: theories of change, indicators & assessment methods

bond.org.uk


cacy

Northern advo-cacy

Service deliv

ery

Hum

anita

rian

resp



Rights

Women’s


sanitation

Hea

lth

Pro

ve &

Impr

ove

It!

Fram

ewor

k

bond.org.uk

Moving forward with PIF

16

Set up way of working

reference groups and a

reference group on thematic areas

Develop back ground

reports with input of

reference groups

Work with FWG to develop coherent

framework

Draft online framework

ready to pilot

Piloting of framework

Final framework

Feb July Oct Nov

Case studies; ‘how to guides’; position papers;

support piloting of new assessment approaches eg

VfM across different ways of

working

bond.org.uk 17

PIF unpacked: a few final points• PIF will not provide an off the shelf solution to assessing and

demonstrating effectiveness, but will provide a very helpful starting point

• Organisations will need to interact with, adapt and contextualize it

• PIF is not presenting anything new, BUT, it is presenting what we already know in one place AND generating agreement on what is best practice

• Will support organisations in engaging with the effectiveness agenda with more confidence

• Will generate greater consistency across the sector in how we assess, manage and report effectiveness

NNGO

1.Delivery of training, mentoring, coaching, consultancy services and support as part of specific programme

capacity development

Capacity development of southern partners and w

ider civil society

More control

More att

ributable to NGO

Less controlLess att

ributable to NGO

Activities and outputs O

utcomes

Impact

11. Significant change s in people’s

lives

Changes in capacity of organisation

9. Improved practices and services of

organisation

10. Improved practices of wider set of stakeholders

8. Strengthened technical capacities

6. Strengthened learning & organisational

capacities 7. Strengthened financial stability

5.Strengthened specific internal processes

4. Strengthened relational & adaptive management

capabilities

2. Delivery of training, mentoring, coaching, consultancy services and support as part general organisational capacity

development

3. Changes in skills, attitudes, behaviours &

actions of individuals / staff

Education Health Livelihoods Rights Women’s empowerment

Water & sanitation

Local capacity

providersSouthern

NGOsSouthern

govt’s

Political, econom

ic, environmental factors / activities of other actors

Sphere of interest: periodically assessed through im

pact assessments

Sphere of influence: changes at this level tracked as part of routine performance m

anagement

system

3

bond.org.uk


cacy

Northern advo-cacy

Service deliv

ery

Hum

anita

rian

resp



Rights

Women’s


sanitation

Hea

lth

Pro

ve &

Impr

ove

It!

Fram

ewor

kThe

Core areas in which NGOs seek to bring

about change; how we categorize of our results

The distinctive contributions that NGOs

make to international development; our added

value

bep quarterly meeting: an emerging framework for proving & improving effectiveness

Documents