Outputs, outcomes and impactsUsing Theory of Change to plan, develop and evaluate your initiative

•To make explicit the link between vision, planning and evaluation

•To introduce you to Theory of Change as a tool for understanding, developing and evaluating your initiative

•To help you prepare for your team planning time this afternoon

2

Aims of the session

Theory of Change is…

•“an outcomes-based, participatory method…for planning, evaluation, and organisational capacity-building”; it “defines all building blocks required to bring about a given long-term goal”(www.theoryofchange.org)

•a development of ‘programme theory’ and ‘logic models’, adapted to complex change where both the path and the destination are evolving (Gamble 2008)

•useful as a “bridging tool” that creates “provisional stability”, linking planning and evaluation (Saunders et al 2005)

3

Introduction to Theory of Change

Purpose of Theory of Change

4

1. Identify your long-term goal.

2. Conduct ‘backwards mapping’ to identify the intermediate outcomes (or preconditions) necessary to achieve that goal.

3. Identify the activities that your initiative will undertake to create these outcomes.

At the same time consider contextual factors, assumptions, resources needed etc…

5

Developing your Theory of Change

A good Theory of Change should be:

•Plausible – the logic of the theory is credible

•Doable – achievable with the resources (and time) available

•Meaningful – stakeholders see the goals as important and worthwhile

•Testable – there are credible ways of discovering whether the predicted results occur (and how/why)

Developing your Theory of Change

6

•In teams, develop a simple Theory of Change for your initiative using the template provided (40 mins)

•Consider:

•Is your goal meaningful?

•Does the logic of your theory make sense?

•What assumptions are you making?

•What resources will you need?

•How will you test your theory?

Activity

7

8

Testing your Theory of Change

1. Data from evaluation activities

•surveys, focus groups, interviews, case studies etc

•time-consuming to collect and limited in scope

2. Data generated by initiative itself

•planning documents, decision logs, network maps, meeting notes, emails etc

•freely available but of limited worth externally

3. ‘Naturally occurring’ institutional data1.NSS, module feedback, retention/achievement data

etc2.valued as impact data but difficult to show causality

9

Sources of data

1. What aspects of your initiative will you evaluate?•Not everything can or should be evaluated – what is both

important and testable?

2. Why are you evaluating?•To meet performance indicators? To show impact? To

understand, inform and develop your initiative?

3. Who are you evaluating for?•Senior managers? Funders? Staff? Students? The

sector? Yourselves?

4. How will you evaluate?•What sources of data will you use? What methods will

you use to collect and analyse the data?

10

Developing your evaluation strategy

•Theory of Change makes explicit what is assumed or tacitly understood about your initiative

•Theory of Change is a planning tool that helps you evaluate and an evaluation tool that helps you plan

•Your Theory of Change is just that – a theory: it should be revisited and revised throughout your initiative

•Use the team planning time this afternoon to develop your ToC further and consider your evaluation strategy (purposes, audiences, data and methods)

11

Summary and next steps

Dozois, E, Langlois, M & Blanchet-Cohen, H (2010) DE201: A practitioner’s guide to developmental evaluation, J W McConnell Family Foundation.

Gamble, J (2008) A developmental evaluation primer, J W McConnell Family Foundation.

Rogers, P J (2008) Using Programme Theory for Complicated and Complex Programmes, Evaluation, 14 (1), 29-48.

Saunders, M, Charlier B, Bonamy, J (2005) Using evaluation to create ‘provisional stabilities’: Bridging innovation in higher education change processes, Evaluation, 11 (1), 37-54.

Theory of Change Community, http://www.theoryofchange.org/

12

References and further reading