1

Module 6: Clear link to results and monitoring of capacity

development

Effective Capacity Development

From Theory to Practice

This module discusses

• How to define capacity results

• How to monitor capacity results

CD Quality grid requirement: 3. Clear link to results and expected

outcomes3.1 Are results and/or outcomes defined beyond what the TC will deliver? Please give examples of key targeted results at the level of enhanced capacity of organisation(s), if relevant outputs that the organisation(s) produce or at the level of outcomes.

3.2 Have the country partners' inputs, which are required to sustain the results of the programme, been identified and specified?

3.3 What innovative forms of TC support have been considered, e.g. regional sources, staff exchanges or networking?

Why Is this Criteria Important?

• Thinking about results is at the heart of good design and management

• Capacity results are as important as other development results but easily lost in the results chain

• How to capture Capacity Results?

• Think in terms of the outputs (enhanced or modified) of the organization/system arising from CD process

• Think beyond what external support will achieve…since has only limited bearing on outcome

Contextual factors and actors within influence

Contextual factors beyond influence

Wider impact

Outcomes

Outputs

Capacity

Recurrent inputs

CD processes

CD

support

Internal resources

CD DEVELOPMENT OBJ (IMPACT)

Increased use of services

CD IMMEDIATE OBJ (OUTCOME)

Performance improved

CD OUTPUTSCapacity: Knowledge, systems,

rules, behaviour

CD PROCESS

CD ACTIVITIESCourses, mentorship, advise

CD INPUTSTA, funds, training

PARTNER ORGANISATION/ SECTOR

OUTCOMEEG: Use of products and

services by clients/ public

IMPACTEG: Improved development

result eg: better health

OUTPUTSEG: products and services

CAPACITYEG: PFM, leadership, logistics,

technical

INPUTSEG: Budget & Staff

Some Challenges

• Easy to fall into trap of focusing on donor inputs and outputs, not organisation/ sector outputs

• Difficulty to specify the performance and capacity changes that are sought, and to propose indicators

• Theory of change often un-developed/ incomplete. Assumption of simple linear causality, and underestimation of impact of different factors on results.

• Pressure to deliver tangible products, less on facilitating processes, and building sustainable capacity for tomorrow

Specifying CD Results

• Link changes in capacity to proposed changes in performance• Performance = Performance change in organisational/

sector outputs (products and services)• Capacity =Enhanced or changed organisational / sector

capacity

• Aim at a realistic balance between:• improvements in capacity that can be achieved over

the short to medium term and • the level of performance that can be expected to

accrue.• Ideally reflected in a corporate/ sector development plan

Look for results beyond TC deliverables

CD suppo

rt

CD processes

Recurrent inputs

CapacityOutputs Outcome

sWider impact

Internal resour-ces

Message 1

Be realistic about what can be achieved:

• Developing capacity is in most cases harder to achieve and support than envisaged

• CD and support processes take longer time than envisaged

• CD support only effective when aligned to partner’s reform process

• Think carefully about the change process required to achieve desired results; is it simple, complicated, complex (next module)

Message 2

Avoid over-specifying results given complexity/uncertainty of many CD processes

• Need to be able to bargain and re-strategise along the way….

• Changing needs arise from lag between design & implementation

• Imperfect knowledge when enter the process

• Be clear on what you want to achieve, leave space to determine how you get there (programme estimates?)

12

Guiding questions for formulating capacity results

• Who or what (organisation, target group, sector, etc.) needs capacity? • Why is the capacity needed – for what purpose? • What type of capacity is needed in order to achieve the purpose?

• Context and Capacity Assessment (QC1) = key sources of information in formulating capacity results

• Quality of dialogue and level of ownership (QC2) and change readiness will influence what can be realistically achieved

Remember….

• This is not about specifying outputs of external assistance

• The results chain firmly belongs to the host

sector/ agency

• The role of external assistance comes later

Exercise:

Make the partner country CD results chain(what are the changes in capacity aimed for):

•CD outcome•CD output(s)•Organisational/Sector Capacity results•CD process results•CD inputs

Part 2: Monitoring Capacity Processes and Results• Monitoring focuses primarily on the processes,

organisation and outputs of capacity development

• Evaluation pays more attention on the CD results produced and their effect on outcomes and impact

• M&E framework is the flip side of the results framework and should be developed AT THE SAME TIME

• ESSENTIAL: M&E is the basis for lessons learning and the justification for changing project structure.

M and E

CD support

CD processes

Recurrent inputs

Organisational/Sector Capacity

Outputs Outcomes

Wider impact

Internal resour-ces Primary focus of

evaluation

Primary focus of monitoring

Monitoring Changes in Capacity

• Purpose is to track changes in capacity over time both for learning and accountability

• Can be done by looking at:

• changes in organisational/ sector outputs and

• changes in organisational/ sector capacity

Changes in Organisational/ Sector Outputs

TC suppo

rt

CD processes

Recurrent inputs

CapacityOutpu

ts

Outcomes

Wider impact

Internal resour-ces

Changes in organisational/ sector outputs

• Delivery of products and services as foreseen in results framework

• A helpful proxy indicator but;– Capacity does not translate into performance

immediately– Outputs can improve for reasons other than capacity

enhancement

• Indicators include: – producer data: services, products, regulations– client / customer satisfaction

Changes in Organisational/ Sector Capacity

TC suppo

rt

CD processes

Recurrent inputs Capacit

yOutputs Outcome

sWider impact

Internal resour-ces

Changes in organisational/ sector capacity

• can be monitored in many ways:– Conduct periodic (self) assessments against a baseline– Conduct Assessments that are Peer-based or

benchmarked against recognised standards

• Indicators need to be jointly selected and understood, not imposed

• Both quantitative and qualitative indicators required to capture both tangible and intangible elements of capacity

• Consider use of innovative methodologies such as MSC, outcome mapping, and appreciative enquiry

Monitoring the Quality of the Change Process

CD suppo

rt

CD processes

Recurrent inputs

CapacityOutputs Outcome

sWider impact

Internal resour-ces

Monitoring the Change Process

• Success depends on joint effort so mutual performance monitoring is important

• What the country partner has done in terms of leading the process• What the external partner has done in terms of supporting the

process

• Can be applied at program level but also at level of individual expert, action

• Need to also take account of changing contextual/ PEA factors during review

• Arrange as periodic performance dialogue using a structured format

• Use of QC in EAMR and ROM

Examples of lead questions

How has the context changed during the course of implementation? (Risks and Assumptions)• Have enabling and constraining factors increased/ decreased?• Has the program responded to a changing context?

 Have the right processes been used to achieve the desired

changes in capacity?• Is the external assistance appropriate and performing?• Is the partner change management appropriate and effective?

 Are results being achieved in terms of enhanced capacity

and performance?• What evidence is there of improved capacity and performance ?• What is aggregate effect of strengthening different components of

capacity on overall capabilities?• Has adequate time elapsed to confirm that change is occurring?• What factors beyond program interventions are influencing results?

25

Who Does What - Monitoring rolesWho does

what?

Monitoring of:

The partner The EC

Inputs & processes

... periodic, systematic dialogue

about mutual performance with TC

personnel

...together with partner; periodic

dialogue about summary of

performance assessment; brokering of

conflicts, adjustments of timeframes

Outputs & outcomes

...has main responsibility; can be

government but also CS watchdogs,

parliament etc. Serves learning,

management and domestic

accountability

...help ensure that results are

monitored appropriately; that results

of CD monitoring are fed back in

adjustments to CD programmes or

operations

Outcomes & impact

...has prime responsibility;

government but also think-tanks,

advocacy groups, universities;

serves learning and domestic

accountability

...mainly to ‘monitor the monitoring by

the partner’ with focus on the capacity

of the domestic monitoring system

(and support it)

Exercise:

Formulate CD indicatorsbased on the results chain

developed earlier

END