Transcript
Page 1: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

i

ActionAid Denmark Evaluation

Synthesis Report

Cowan Coventry

13th January 2012

INTRAC Oxbridge Court, Osney Mead, Oxford, OX2 0ES UK www.intrac.org

Page 2: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

ii

List of abbreviations AADK ActionAid Denmark

AAI ActionAid International

BLD Building Local Democracy

CEO Chief Executive Officer

CSO Civil Society Organisation

EC European Commission

ELBAG Economic Literacy and Budget Analysis in Governance

HRBA Human Rights Based Approach

IGT International Governance Team

JDG Just and Democratic Governance

MOU Memorandum of Understanding

MS Mellemfolkeligt Samvirke

NDS National Development Strategy

P4C People for Change

RfD Recycling for Development

TCDC Training Centre for Development Cooperation

TOT Training of Trainers

T4C Training for Change

WFI Women for Influence

Page 3: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

iii

Contents

Executive Summary ........................................................................................................... iv

1. Introduction .................................................................................................................. 1 1.1. Background to the evaluation ................................................................................................. 1

1.2. Methodology .............................................................................................................................. 2

2. Relevance and effectiveness of AADK programmes ................................................. 3

3. AADK in the Danish context ........................................................................................ 7

3.1. Sectoral trends in the international and Danish context.............................7

3.2. Fundraising .............................................................................................................................. 12

3.3. Media and campaigning ........................................................................................................ 14

3.4. Global Contact ........................................................................................................................ 16

4. AADK in ActionAid International............................................................................... 19

4.1. Leadership of the Just and Democratic Governance theme ........................................... 20

4.2. The Activista programme ...................................................................................................... 22

4.3. Training4Change .................................................................................................................... 24

4.4. People4Change ...................................................................................................................... 27

5. Managing Change in AADK ....................................................................................... 30 5.1. Revised governance .............................................................................................................. 30

5.2. Change management ............................................................................................................ 32

6. Conclusions ............................................................................................................... 33

Annex 1: Objectives and scope of the evaluation ......................................................... 37

Annex 2: Evaluation surveys: methodologies ................................................................ 39

Page 4: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

iv

Executive Summary

1. MS-ActionAid Denmark (AADK) has embarked upon a wide range of transformative

changes since 2006 including refocusing its programme work on the governance

theme; redesigning its development worker programme as People for Change;

revamping and expanding its training work through Training for Change and a

network of Global Platforms; and perhaps most fundamentally deciding to affiliate

with ActionAid International. The purpose of this evaluation is to assess the

relevance of the strategic decisions taken by AADK in the last six years and its

effectiveness in implementing them.

2. The evaluation was based upon a comprehensive data gathering process which

included twelve surveys, including two tracer studies; three field visits; and

approximately 200 interviews. Seven separate, stand-alone evaluation reports and

two internal reports have been produced on AADK programmes and operations. This

synthesis report summarises the key findings from these and assesses more broadly

the overall relevance and effectiveness of AADK’s strategic decision-making and the

implications for future strategy development.

3. In 2011, prior to this evaluation, AADK commissioned a number of summative

evaluations of their former country programmes prior to their merger with AAI. The

evaluations confirmed the relevance of the ‘democracy focus’ of AADK country

programmes and highlighted a number of achievements in the different governance

themes. Some key recommendations were made with regard to future AADK

governance work - for example, for better linkages between local and national level

governance work; for support to both the demand and supply side of governance;

and for improved M&E systems to better measure the effectiveness of governance

activities.

AADK in the Danish context

4. AADK strategic choices and programming in recent years - for example, its decision

to focus on governance, its work on strengthening capacity through T4C and P4C

and connecting people through Global Contact - have all been well-aligned with the

Danish Civil Society strategy. Danida is currently in a period of transition with a new

government and its Civil Society strategy under review. While AADK is mostly well-

positioned with regard to emerging trends it would be prudent to anticipate that

Danida’s policies and practices vis-à-vis civil society may evolve in line with those of

other Northern donors. This might translate into major changes in traditional funding

arrangements for NGOs including reduction in net funding available; an opening up

to more partners; increases in the match funding required from partners; and more

explicit links to the demonstration of results.

5. AADK’s public fundraising was propelled by Danida’s stipulation in 2006 that it

provide 10% of the Frame Agreement budget from its own resources. AADK has

significantly grown its gross income from public fundraising from a very low base.

Although it has been able to meet its match funding requirements, net income has

stagnated in the last two years. It is unlikely to meet its fundraising targets without

significant investment. AADK finds itself in a difficult situation - to increase its general

funds it will need to invest, but all the income it raises is already allocated to

programmes under the Frame Agreement. AADK would be more likely to reduce its

Page 5: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

v

dependency on Frame funding if the Agreement allowed it flexibility to use its own

matching funds to invest in programme activities outside the agreement including

marketing and fundraising.

6. AADK is committed to building a campaign organisation in Denmark based on youth

activism. This focus on popular activism to raise public awareness on development

issues draws upon its history of ‘connecting people’ and reflects the focus and

constraints of Danida funding for public education on development issues in

Denmark. Despite limited resources, AADK has achieved a good campaigning profile

and strong media coverage in recent years that have contributed to AADK being

seen as a credible, authoritative voice in the Danish press. AADK would be even

more effective in campaigning, however, if it adopted a more strategic, outcome-

oriented approach. This would focus on policy and practice change for the benefit of

poor people in the South as the ‘end’ purpose to which mobilising activism is one of

the ‘means’. A key step will be to establish planning mechanisms to coordinate

across the organisation and to prioritise campaigns. It may be about ‘doing less, but

doing better’.

7. Volunteers, host organisations and families express high levels of satisfaction with

Global Contact. There is evidence that volunteers gain a more global perspective on

development issues and host organisations report both tangible and intangible

benefits from the experience. The data is too limited to draw firm conclusions on the

linkage between volunteering and increased social activism. There are some positive

indications, but it would require more systematic monitoring of volunteers over time to

explore this further. Global Contact also benefits AADK since volunteers become

AADK members and the programme provides a significant proportion of the income

of Global Platforms through course fees. AADK should dialogue with AAI to explicitly

explore whether Global Contact can contribute to the new strategy as a ‘business’ or

‘social’ programme.

AADK in ActionAid International

8. AADK’s leadership of the Just and Democratic Governance theme in AAI is viewed

extremely positively by AA country programme staff who have contact with it. The

vast majority report that it was very important in helping deliver their governance

work and that they would welcome AADK’s further promotion of and support to them

in the federation. Key among its achievements has been the training of more than a

thousand ActionAid, partner and other CSO staff in governance in 2010/11, creating

a skilled constituency of support for governance work. The evaluation recommends

AADK continue to offer leadership in governance work in the new AAI strategy. There

should, however, be a clear mandate and effective system of accountability to the

AAI Secretariat. As an interlocutor for Danida funding, AADK should develop ways to

demonstrate clear added value and benefit to Southern civil society.

9. AADK took over the coordination of the Activista, a global youth activist programme

in AAI, in 2009. AADK has revitalised the programme, with limited resources. The

global outreach has contributed to its attractive youth profile in Denmark. The

mobilisation of youth in support of pro-poor change features prominently in the new

AAI strategy in key promise 6 (to mobilise five million youth). The specific contribution

of Activista to this promise is not altogether clear. The evaluation recommends that

AADK continue to coordinate Activista, subject to clarification of the role of Activista

in the new strategy and the level of resourcing available to it. They should also

consider the possibility of relocating its coordination to the South in the next strategy

Page 6: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

vi

period.

10. AADK’s Training for Change (T4C) programme is centred around the well-

established TCDC training centre in Tanzania and seven smaller training centres or

‘Global Platforms’. T4C has demonstrated its popularity and effectiveness since its

inception by delivering a range of training products to a growing client base (5,000

participants at TCDC, 2,500 at Global Platforms) who rate the quality of training

highly. TCDC training products are increasingly linked to the capacity building needs

of AAI and partner staff in the theme of governance. TCDC and the Global Platforms

are heavily dependent on Danida funding and income from Global Contact

respectively. T4C plans to expand its operations in the next few years. The

evaluation recommends that any future growth be based on consolidating the quality

of what it has achieved to date by developing a theory of change and business plan

that more clearly identifies its target audiences; consolidates the quality of products

offered to them; and identifies new income-generating services. It recommends that

T4C continues its collaborative relationship with AAI, but continues to be managed by

AADK so that it can build a sustainable business model based on its flexibility to

respond to the needs of diverse stakeholders.

11. There is clear evidence that the People for Change (P4C) programme, through

Inspirators and Advisors, has provided value-added to partners and contributed to

their capacity development in the relatively short period since its inception. The

emphasis on a people-to-people approach to capacity building is valued by

participating organisations and there is evidence of the programme contributing to

the delivery of governance-related programmes. The evaluation concludes that

AADK should work to improve the profile and level of understanding of P4C in AAI in

the next phase and that P4C should form an integral part of an AAI capacity

development framework. It recommends that P4C remains a delegated responsibility

of AADK within the federation and continues to consolidate its focus on work on

governance issues.

12. The evaluation was unable to assemble, to its own satisfaction, consolidated financial

management information relevant to a thorough review of the ‘business models’ of

T4C and Global Contact. More needs to be done to ensure that the Leadership Team

has complete financial management data to enable it to understand and monitor the

underlying finances of AADK and its operations.

Conclusions

13. The evaluation concludes that the strategic choices made by AADK in recent years

were relevant and appropriate. For example:

The decision to adopt a ‘democracy focus’ in its programming has been

vindicated by the subsequent evaluations of country programmes; enabled AADK

to demonstrate a specific added value in programming to both AAI and Danida

through its leadership of the AAI Just and Democratic Governance theme; and

contributed to the adoption of AAI’s new strategy objective ‘Holding Governments

to Account’.

The launch of People4Change has enabled AADK to evolve from the outdated

concept of a development worker model to experiment with a demand-driven,

flexible model of development cooperation that resonates with a growing interest

in the North and South with people-to-people collaboration.

Page 7: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

vii

The Training4Change programme has demonstrated its popularity and

effectiveness by delivering an expanded range of training products to a growing

client base who rate the quality of training highly.

Volunteers, host organisations and families express high levels of satisfaction

with Global Contact which continues to translate AADK’s traditional values of

inter-cultural understanding and solidarity in a contemporary setting

The decision to assume leadership of the Activista programme in AAI revived the

project and has reinforced its youth activist profile and campaigning capability in

Denmark.

14. The evidence from interviews and the staff surveys also suggests that the majority of

those consulted broadly feel that the right decisions were made; that senior

management implemented the change process openly and fairly; and that the

organisation is heading in the right direction.

15. The evaluation also identifies a number of continuing challenges that remain to be

addressed in the new AADK strategy. These focus around AADK’s interface with

Danida; Danish civil society, including its own membership; AAI; and Southern civil

society.

16. AADK will need to be sensitive to trends in the sector in relation to the Paris Agenda

and their possible implications for its Frame agreement. The new strategy should

consider additional funding models; how to report most effectively on its programmes

to demonstrate their benefits to Southern civil society beyond AAI. This will require

revisions to current management and information systems.

17. AADK offers two distinct profiles in Danish civil society. On the one hand, it has

successfully developed an agency profile associated with youth activism. On the

other, it is seen as well-established, ‘reputable’ development NGO and draws upon a

constituency of support among the middle-aged. While AADK dialogues with the

’young’, it fundraises from the ‘old’. AADK will need to manage its programme

strategies and public communications sensitively to ensure that these different

profiles do not work at cross purposes.

18. The new AAI strategy offers AADK a potential mandate and opportunity to extend the

geographical range and impact of its programmes. However, management

arrangements outlined in the MoUs between AAI and AADK on governance,

Activista, P4C and T4C have been less than fully effective. It will be important that

any future delegated responsibility to AADK as a Member be accompanied by the

effective implementation of a stronger mandate from and system of accountability to

AAI.

19. AADK’s relationship with Southern civil society is changing as a result of the merger

of MS and AA country offices. There is also some evidence that AADK programmes

such as T4C and P4C are increasingly orienting themselves as service providers to

AAI country offices. AADK will need to continue to demonstrate its programme links

with and benefits to independent, diverse Southern civil society to support its public

fundraising; to feed and legitimate its campaigning; and to meet the needs of its

major donor, Danida.

20. In conclusion, AADK has demonstrated a track record of achievement during a

period of wide-ranging and profound change. It remains in a period of transition. The

Page 8: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

viii

new AAI strategy offers a number of opportunities for AADK to increase its impact

and influence. The evaluation believes that AADK’s strengths are focused around

two programme approaches to delivering its core competence in governance:

Connecting people, particularly young people, to build a global civil society to

fight poverty

Strengthening capacity of Southern civil society e.g. T4C, P4C to help make

governments more accountable to poor and marginalised people.

Both these ways of working are relevant to AAI strategy ’Peoples Action to End

Poverty’ and to the strategy objective of ‘Holding Governments to Account’. AADK

will be at is most effective when it retains a clear focus and maximises the synergies

between the two approaches.

Page 9: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

1

1. Introduction

1.1. Background to the evaluation MS-ActionAid Denmark (AADK) has undergone a period of profound and wide-ranging

change since 2006. The many changes include:

A decision to focus its programme work exclusively on governance;

A re-design of its capacity building approach asTraining4Change;

A replacement of its development worker programme with the People4Change

programme, introducing the new concepts of ‘Advisors’ and ‘Inspirators’;

Investment in improving its public fundraising in order to meet the 10% matching

funding from public donations required by its Frame grant from Danida.

A revised system of governance, reverting to a representative Council in 2009 after

having introduced a General Assembly in 2005.

A re-profiling of itself as a ‘youth activist’ organisation through its media and

campaigning work.

Perhaps the most significant change was its decision to become a member of the global

NGO federation, ActionAid International (AAI). AADK became an Associate Member of AAI

in 2008 and a full affiliate as ActionAid Denmark (AADK) in July 2010. MS country offices

have now been merged with ActionAid International country offices. In association with the

process of affiliation with ActionAid International it was agreed that AADK played a

leadership role in two areas within the federation:

The Just and Democratic Governance theme

The Activista project, a global youth campaigning network.

In addition, AADK agreed with ActionAid International to continue to run the

Training4Change (T4C) and People4Change (P4C) programmes (including Global Contact

and the Advisors and Inspirators programme) while exploring possible ways to integrate

these into the ActionAid International structure.

AADK is due to discuss its pending application for Frame funding later this year and,

following the publication of ActionAid International’s 2012-17 strategy “ People’s Action to

End Poverty”, to begin its own strategy development early in 2012. In preparation for both

events, it decided to commission a comprehensive evaluation of the changes introduced in

recent years. The Terms of Reference (TOR) (see Annex 1) listed four objectives for the

evaluation:

Assess the relevance of the decisions and directions taken by MS/AADK over the

last six years vis-à-vis general developments in the external environment, including

the Danish CSO strategy.

Assess to what extent AADK has succeeded in implementing its various strategies

and meeting annual plans and targets.

Provide input to the internal discussion within AADK and to AADK’s dialogue with

AAI and with Danida on the strategic and programmatic direction of AADK (the new

strategy) over the coming six years.

Provide evaluation inputs as agreed in the MoUs with AAI within the fields of T4C,

P4C, the Governance theme and Activista.

Page 10: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

2

AADK commissioned INTRAC in September 2011 to conduct the evaluation. The core team

of evaluators, following inception visits to Copenhagen, produced an inception report on 13th

October 2011. This summarised the methodologies to be deployed and focus of enquiry for

each of the component elements of the evaluation as outlined; and confirmed the written

outputs to be delivered for each of these. This comprehensive evaluation report draws upon

seven stand-alone reports on:

Training4Change

People4Change

AADK leadership of the International Governance theme

AADK support to the Activista programme

Global Contact

AADK Media and Campaigning

AADK Fundraising

In addition, written outputs have been produced on the other elements of the evaluation and

incorporated into this report i.e.

Role of AADK in Danish and international civil society (Section 3)

Assessment of AADK management and governance (Section 5)

This evaluation report addresses the objectives of the TOR in each of the key programme

areas and business units. It reviews the relevance of the strategic decisions taken; the

effectiveness in the implementation of those strategies; and the implications for AADK’s own

strategy development. These will be ‘clustered’ under three headings:

AADK in the Danish context (Section 3)

AADK in ActionAid International (Section 4)

Managing change in AADK (Section 5)

The implications for AADK’s own strategy development and relationship with its major donor

Danida will be considered more broadly in Section 6.

1.2. Methodology

This report is the product of an intensive process of data-gathering during October and

November 2011 for the eleven elements of the evaluation. A range of methods was used to

gather and analyse data for the different work-streams. These are:

Reviews of relevant documentary materials.

Approximately 200 semi-structured interviews with a range of stakeholders, details

of whom can be found in the separate evaluation reports. .

Data gathered from the field research in six countries. A field visit was made to

Tanzania and Kenya to gather data for the Training4Change evaluation; to Zambia

for the People for Change evaluation; and to India and Nepal for the Global Contact

evaluation. A local consultant in Myanmar gathered data on the AA Fellowship

Programme as a potential comparator for the P4C programme.

Twelve surveys, including two tracer studies, each addressing specific groups of

respondents and contributing to different elements of the evaluation. Annex 2

provides a full description of the research purpose of the surveys, their design,

Page 11: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

3

sampling methods, and data analysis. The following table provides a summary of the

survey types and responses:

Survey title Sample size No. of responses Response (%)

Tracer study: Global Contact volunteers 190 61 31.5

Tracer study: Participants in Global

Change training

95 33 34.7

Survey: Global Contact host

organisations for volunteers

31 16 51.6

Survey: Participants in TOT courses at

Global Platforms

74 29 39.2

Survey: Organisation that had sent staff

on or commissioned governance

courses

142 22 15.5

Survey: Participants on governance

courses

485 111 22.8

Survey: P4C Advisors 29 18 62

Survey: P4C Inspirators 67 38 56.7

Survey: P4C partner organisations 95 65 68.4

Survey: Country Directors 33 10 30

Survey: Governance focal points 55 15 27.2

Survey: AADK former and current staff 29 28 97

The short time frame available to the evaluation team presented a number of challenges in

managing extensive data gathering processes. Some limitations affecting the data collection

and analysis should be noted in relation to the plausibility of the findings and the ability to

generalise from them (these are explored more fully in Annex 2). While response rates were

low they can be considered as representative. However, there is a potential for bias due to

convenience sampling in some cases. A tight timeframe for the administration of the surveys

had a number of consequences. It did not allow for more robust piloting and redesign of

survey questions and respondents with invalid email addresses had to be excluded rather

than further investigated. The time-span for the tracer studies between Then and Now was

short. Perhaps most importantly data gathering through interviews was taking place at the

same time as the application of the surveys so that survey data could not be followed up

systematically through other data collection methods such as interviews.

The evaluation team met on 24th and 25th November to triangulate the primary data gathered

through the surveys and tracer studies with the data gathered through fieldwork and

documentary review, and to form an overall analysis to inform this report. Initial findings were

shared with the AADK Leadership team on 28th November and comments and suggestions

incorporated into first draft reports delivered to AADK on 9th December 2011. This final

synthesis report has sought to incorporate comments received on the first draft report/s.

2. Relevance and effectiveness of AADK programmes

AADK commissioned a number of programme reviews/evaluations during 2011 with a view

to key findings and learning feeding into this broader, comprehensive evaluation. AADK

commissioned summative evaluations of each of the nine former country programmes

(whose cooperation agreements were due to expire at the end of 2011). A synthesis report

Page 12: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

4

summarising the main lessons learned was produced in July 2011. A review of the AADK

regional MENA Youth Programme was also produced in August 2011.

Drawing upon these reviews, this Section assesses the relevance and effectiveness of the

programmes under review and summarises the key lessons to emerge from these with

relevance to this evaluation. Separate attention is given to the review of the ‘Recycling for

Development’ (RfD) project and its strategic coherence with other AADK activities.

2.1. Country programmes on governance

AADK’s 2008-12 Country Programme Strategies (CPS) for the nine country programmes

were developed within the guiding ‘Democracy Focus’ for AADK country programmes. This

consisted of five themes - Building Local Democracy (BLD), Land Rights, Anti-Corruption,

Conflict Management and Peace-building and Trade Justice - associated with governance

work more broadly. CPSs were developed to a standard template to ensure some degree of

coherence in programming. Each country programme had to allocate 50% of its budget to

the BLD theme while 30% could be allocated to a maximum of two other themes. Countries

were free to allocate the remaining 20% on country specific or innovative work.

All the above evaluations confirmed the relevance of the ‘Democracy Focus’ of the country

programmes and recommended their continuation. AADK partners were seen generally to

be relevant to the thematic work undertaken although a significant number of partners were

inherited from the previous strategy period and had no particular track record in governance

work. However, since they already had a rights-based approach to development they

adapted well to governance with AADK’s capacity development support. AADK’s approach

to partnership - characterised by mutual respect, regular communication and support - was

reported as being relevant, efficient, effective and highly appreciated by partners.

All the evaluations reported a growing AADK competence in governance work and a number

of specific achievements under each of the governance themes.

The synthesis report highlighted a number of changes in the lives of rights holders under the Building Local Democracy theme. For example:

Increased citizen awareness of governance processes e.g. through popular

booklets and posters in Uganda, Sudan and Tanzania.

Increased citizen participation in governance structures such as Village

Development Committees (Nepal) and School Committees (Tanzania).

Increased accountability of local budget expenditure through public expenditure

tracking in Tanzania, Uganda and Sudan.

The creation of citizen platforms such as youth fora in Zimbabwe and civil society

fora in Uganda that were subsequently trained in, for example, empowerment,

accountability, planning and budgeting, and advocacy.

Increased involvement of youth and women in governance processes e.g. women’s’

participation in local government planning and budgeting in Uganda and the

mobilization of youth at village, ward and district levels in Tanzania.

Anti-corruption was a major theme in the Uganda Kenya and Mozambique country

programmes and as a cross-cutting theme in Sudan, Zimbabwe and Tanzania. The

evaluations reported an increased public awareness of corruption (Uganda); reporting of

cases of corruption by members of the public (Kenya); and the creation of coalitions to fight

corruption (Uganda). The synthesis report noted also some of the challenges associated

Page 13: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

5

with anti-corruption work such as the threat to the personal security (Uganda and Kenya),

and limited space for criticism (Mozambique).

The Tanzania, Zambia and Nepal programmes included work on the Land Rights theme. In

addition to contributing to an increased awareness of land rights among marginalised

populations, the evaluations reported increased involvement of community members in land

management committees and an increase in national-level advocacy on pro-poor land

policies. Perhaps most importantly, some specific examples were recorded of poor and

marginalised people acquiring land ownership rights (e.g. in Nepal and Zambia).

The Conflict Management and Peace-building theme succeeded in establishing and

training members of peace or conflict-resolution committees in country programmes - for

example, local peace committees at village and district level in Nepal and up to 60 conflict

resolution committees in Zimbabwe - and provided examples in all countries of conflicts

being resolved as a result.

Three countries - Uganda, Zimbabwe and Guatemala - focused on Trade Justice and on

improving the access of poor, marginalised people to markets and enabling them to get fair

prices for their products. The evaluations reported widespread support in developing the

capacity of poor farmers, women farmers in particular, to advocate for better prices for their

products. The Zimbabwe evaluation reported that AADK training was a key factor in enabling

partners to successfully negotiate better prices and improve markets for their products.

The evaluations also highlighted a number of issues/lessons of relevance to country

programming. Some of the learning points were of a ‘technical’ nature regarding the design

and implementation of governance programmes e.g. the use of communications media,

gender analysis, training and learning materials. However, some ‘generic’ issues were raised

that continue to be relevant to AADK support to governance work at country level. For

example:

Capacity development

Most evaluations recognised AADK’s important role in strengthening the capacity of partner

organisations to contribute to governance work through a broad suite of approaches

including training, workshops, coaching and mentoring by development workers and

programme officers, and peer-to-peer support. The Nepal evaluation suggested that this

support had tended to be ad hoc rather than on the basis of a systematic organisational

assessment process. The Synthesis report highlighted that in several evaluations partners

expressed a preference for AADK capacity building support to be demand-driven and for it to

adopt more of a coaching than a formal training approach. The Tanzania evaluation noted

that partner organisations, once their own capacity had been strengthened, did not always

similarly invest in the training of rights-holders.

Measuring effectiveness.

While the evaluations were able to document the increased capacity of partners, some

evaluations e.g. Mozambique, commented on the difficulty in documenting evidence of

increased influence, for example, on national level policy change. The Tanzania evaluation

noted that M&E systems needed to be improved to better document concrete changes for

rights-holders and suggested using Outcome Mapping in future planning with partners to

better measure changes in attitudes and behaviour. A number of evaluations (Uganda,

Zimbabwe) commented on the need to use base-line surveys, SMART objectives and more

relevant indicators to better measure the effectiveness of programmes.

Page 14: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

6

Governance programming

The evaluations of Uganda, Nepal, Zimbabwe, Guatemala and Mozambique all noted that

better linkages between the themes at country level would have resulted in more programme

synergies. While the strategies generally recognised the need to create linkages between

local activities and national policy issues, some evaluations (Guatemala, Nepal, Zambia and

Mozambique), indicated that local/national links were relatively weak, and that more effective

vertical and horizontal linkages and networking would enable local evidence to feed into

national advocacy. Several evaluations (Nepal, Kenya, Uganda and Zimbabwe) pointed out

that the AADK approach to governance tended to focus exclusively on the demand-side of

the equation i.e. right-holders, and would benefit from a recognition of the need to invest in

building the capacity of the supply side i.e. duty bearers.

Partnership

The respectful, flexible approach to partnership adopted by AADK was much valued by

partners - not only in relation to capacity development but also (e.g. in Guatemala,

Zimbabwe) to sharing and deepening knowledge, supporting networking and alliances.

Concern was expressed in a number of evaluations (Kenya, Zambia and Zimbabwe) that

this partnership approach based on shared values and a common vision needed to be

continued post-merger.

2.2. MENA Regional Programme on ‘Building Bridges’ Since January 2009 AADK has implemented the “Building Bridges for Youth Action

Programme” in the MENA region with Danida funding. The overall objective of the

programme is the enhanced capacity of youth in the region to participate in democratic

reform processes and intercultural dialogue and understanding between youth in Denmark

and the region. The programme has three specific objectives:

Increased youth participation in democratic reform processes and intercultural

dialogue.

Strengthened, accountable youth groups engaged in national democratic reform

processes.

Increased tolerance and new contacts established between young women and men

from the region and Denmark.

An evaluation conducted in August 2011 concluded that the objective of the programme,

following the events of the ‘Arab Spring’, was even more relevant today. In light of the

impressive number of activities and youth involved, it recommended that the programme be

continued for at least another three years.

The evaluation found that the programme was exceptionally effective in promoting

intercultural dialogue and understanding. It was less certain whether the programme had an

impact on reforms in the region and made a number of recommendations with this is mind.

Key among these was for the programme to adopt more of an organisational and less an

individual focus by:

Refocusing the programme on the strengthening youth organisations and on youth

involvement in CSOs rather than the capacity of youth more generally.

Developing a tool to asses capacity of youth organisations and provide training on

the basis of a capacity assessment and plan to use the training.

Strengthen existing, loosely organised youth groups or CSOs rather than focus

mainly on unorganised youth.

Page 15: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

7

Developing a strategy to enable youth groups to build on local issues to engage on a

national level.

Developing strategic partnerships, especially in Jordan and Lebanon, to jointly

develop the next phase of the programme.

2.3. The Recycling for Development project.

The evaluation was also asked to consider a recent review of the ‘Recycling for

Development’ (RfD) project, managed by AADK, and its strategic coherence with other

AADK activities. A draft report was produced in November 2011 and the review has been

extended subsequently to examine strategic linkages to AADK general programmes and

partnerships.

AADK believes RfD offers a valuable outreach into segments of Danish civil society who are

not normally engaged in public debates on development but who nonetheless feel a strong

sense of solidarity and connectedness to their counterparts in South. As such, it believes it

offers the potential to form part of AADK’ solidarity work in connecting people to build global

citizenship.

However, this aspiration should be put in perspective. The draft review report suggested in

November 2011 that the relevance of the project can be considered in relation to its

contribution to mobilising popular support for development assistance in Denmark or for its

contribution to development, particularly civil society development in the South. In relation to

the former, it points out that the constituency of support that RfD draws upon consists mostly

of retired Danish people and non-ethnic Danes or diaspora populations with a low level of

understanding of the rationale for RfD in development terms. RfD seems less attractive to

young people. There is no obvious connection, therefore, with AADK’s people-to-people

programmes such as Global Contact and Activista. Nor is there a demonstrable ‘added

value’ between the project and AADK’s development focus on governance, the P4C and

T4C programmes.

There may be a case for AADK to continue to manage the project in order, for example, to

include diaspora populations in its activities (though it is not entirely clear which activities)

and/or its supporter base (if there is no conflict of interest in AADK doing this while

subcontracted to manage the project). In either case, the review indicates a number of areas

where AADK would have to invest significantly in improved project management to make it

efficient and effective as a development initiative.

3. AADK in the Danish context

This section examines AADK’s profile and activities in Denmark, in particular its fundraising

efforts, media and campaigning work, and work with Global Contact volunteers. First, we

review AADK’s positioning in relation to sectoral trends during the period under review and

the current Danish Civil Society Strategy.

3.1. Sectoral trends in the international and Danish context

Northern NGOs in recent years have faced a time of unprecedented, and perhaps still

underestimated, change. In developed countries, indebtedness, low rates of growth and

public sector cuts have cast a shadow over future prospects for aid funding. In contrast,

many countries in the South are growing rapidly and projecting a new sense of confidence

Page 16: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

8

and energy. The balance of economic and political power between countries and regions is

shifting with the new axis moving away from the Atlantic to Mumbai, Shanghai and Sao

Paolo.

New actors have entered the field of international development. Philanthro-capitalists, such

as Bill and Melinda Gates, have arrived with ambition and money, demonstrating a desire for

a more personal and active involvement in the work they fund. Private sector companies are

now delivering many large-scale development programmes including support for civil society

and are engaging in other ways through agendas of corporate social responsibility.

New relationships are developing, unmediated by traditional organisations and mechanisms.

The recent spontaneous mobilisations of the ‘Arab Spring’ have inspired others, including

the Occupy movement in Europe and America. These movements have been energised

and often led by young people. Their determination and rejection of the status quo have

encouraged others to believe that change is possible and acted as a catalyst for wider

action. Technologies such as the internet and social media have offered opportunities for

different, more networked forms of organising, as well as new ways for individuals to engage

and express commitment. The ‘spontaneity’ of these revolutions at the same time points to

the value that traditional organisations can offer to lasting social change with their analysis,

experience and practical ideas.

These trends are complex and do not offer a single story, but it is clear that they are

potentially changing the way that civil society actors relate and potentially challenge the

future role and relevance of NGOs.

Shifts in major donor policy and practice are having a big impact on NGOs, particularly in the

Nordic countries and Netherlands where they have been historically more dependent on

government funding. Meetings between bilateral donors and developing country

governments since the Paris Declaration in 2005 have revealed an evolving, and at times

contradictory, approach to aid effectiveness, the key elements of which are:

Harmonisation: the coordination of donor of donor efforts at a country level with

programme approaches, joint strategies and programming and various delegated

forms of cooperation.

Ownership: this principally refers to the primacy of national government ownership

but applies also to greater support for ownership by and investment in southern

CSOs.

South- South cooperation: the draft Busan declaration includes explicit mention of

greater support to south-south or “triangular” cooperation and learning.

Moving beyond aid : the draft declaration makes explicit mention of “diverse

modalities” including references to trade, private investment and financial

mechanisms, and climate finance

Moving beyond traditional aid actors: it also refers to diverse actors making

reference to philanthropy, the private sector, parliaments and local government.

Focus on results, transparency and accountability: with particular emphasis on

the quantitative and improving statistical capacity.

Most Northern donors subscribe to the aid effectiveness agenda and the implications for

NGOs are still evolving. Donors continue to fund their national NGOs but there is some

evidence that their current modalities and level of support to NGOs may be affected by a

number of factors. These include:

Page 17: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

9

The financial crisis and the pressure on aid budgets;

The trend to decentralise more assistance to the country level and to channel

support more directly to Southern CSOs;

The desire to fund a more diverse group of organisations

NGOs continue to be an important channel between individual donors and national civil

society organisations, but there is evidence that this is not quite the pre-eminent position

they used to enjoy. Donors are looking at new and different modalities that allow for more

direct support at a country level, as the growth of pooled basket funds to support CSOs in

the South illustrates. A more explicit questioning of the role and value added that Northern

NGOs bring is emerging from donors and from national CSOs themselves.

In addition there is a growing pressure on NGOs to demonstrate results more effectively -

not only as a result of the aid effectiveness debate but also in response to pressure from

national audit offices and criticism of the sector. This has led to a growing emphasis on

‘objective’ measurement of results including quantitative data, random sampling, and value

for money assessments.

Northern NGOs have begun to feel the impact of these trends in recent years. For example,

total funding for Dutch NGOs has been reduced from 550 million Euros to 400 million Euros

p.a. meanwhile funding of local NGOs through embassies has increased to150 million Euros

p.a. In addition, Dutch NGOs in receipt of framework agreements must now provide 25% of

their own funds; this may increase in the next round of funding in 2015. There has been

some reduction in the funding of Frame Agreement partners in Denmark but not to the same

degree as other countries. 10% matching funding must now be raised from NGOs own

sources as a condition of funding. This has presented a challenge to Danish NGOs,

particularly those without a humanitarian mandate, to increase their public profile in a small

and competitive domestic market.

Other donors have not yet adopted such drastic reductions but have renegotiated ‘historical’

funding relationships with NGOs. DFID has recently restructured its Programme Partnership

Agreements, awarding them on the basis of quality of organisational submissions and

alignment with DFID policy. Some long-standing partners have lost core funding and, for the

first time, some Southern organisations have gained core funding under this arrangement.

Donors are also changing the focus of aid and aid modalities. The trend has been to reduce

the geographical scope of aid and to focus on the poorest countries. An increasing emphasis

on ‘fragile states’ reflects an overlap with Northern government continuing security concerns

and a security agenda. In these contexts, the role of civil society organisations is seen as

crucial by donors but carries risks of them becoming instrumentalised. Equally, in more

stable contexts the return of an emphasis on the role of the state, as evidenced for example

in the Paris-Busan process, may reduce CSOs to the status of sub-contractors of ‘gap

fillers’.

Danida support to Danish and international civil society, including its Frame Agreement with

AADK, is governed by its “Strategy for Danish Support to Civil Society in Developing

countries” which is due to be reviewed in 2012. The Civil Society Strategy is committed to

the Paris Declaration as a “new and decisive framework for making development assistance

more effective”. (p 20) The trends outlined above are clearly signposted in the strategy

although NGOs in Denmark have been less affected by them to date than sister agencies in

other parts of Europe.

Page 18: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

10

As is to be expected the Civil Society strategy makes reference to Danida’s intention to

“strengthen the results-orientation of the activities implemented in support of civil society

development” (p 18) and the challenge facing civil society with regard to “willingness and

ability to explain and account for goals and results to members, public authorities and the

general public” (p 10). Danida has been comparatively flexible to date in its reporting

requirements from partners with Frame Agreements. But it is unlikely to be immune from the

increasing pressure in the sector to demonstrate results. Quite what the implications might

be for Danish NGOs is not yet clear.

The overarching objective of the Danida Civil Society Strategy is “to contribute to the

development of a strong, independent and diversified civil society in developing countries” (p

7). The emphasis on an independent civil society is in line with the more general donor trend

to strengthen the ownership and protagonism of diverse national CSO actors. One of the key

roles, therefore, envisaged for Danish organisations is the development of capacity of their

partners in developing countries. The strategy recognises that the growing complexity of

development means that greater demands will be placed on “the professional capacity of

Danish organisations”. They must not simply be a channel for funding but build up “core

competencies” where they can make a difference and help develop “innovative models”.

This type of language suggests that Danish NGOs will increasingly be expected to clearly

define what their added-value is as interlocutors with the South. There is reference also to

the role of Danish organisations connecting people and civil society organisations in

Denmark and developing countries in order to exchange ideas and learning that is of mutual

benefit but this is not fully developed in the current strategy.

In the section on ‘globalised and nationalised organisations’ on pages 23, 24 the strategy

reveals a somewhat ambivalent attitude towards ‘nationalised’ global NGOs. The growing

internationalisation of NGOs through global federations/confederations is a current

preoccupation in Danida and is also mentioned in the TOR for the pre-study. The rationale

for this process of internalisation includes increased impact through globalised networks;

cost-efficiency in country infrastructures; expanded fundraising markets; and increased

legitimacy through the inclusion of Southern affiliates.

It is too early to reach any definitive conclusions about how far international NGO alliances

are meeting their objectives in terms of increased influence and cost-efficiency, but their

impact on the concept and practice of partnership has come under some scrutiny. There is

some evidence that while partners appreciate the reduction in administrative work caused by

a ‘unified’ country presence, in other ways it might reduce their options - particularly in

relation to Nordic NGOs who traditionally have tended to offer long-term engagement and

capacity building of CSOs as an end in itself. Aspects of good practice in partnership are at

risk of being lost as affiliates lose the direct relationship with partners through their own

country offices and staff. The relationship of the Northern affiliate becomes one with the

‘international’ country office which can introduce some confusion over who is the partner and

whose capacity is being built. Danida appears to be keeping an open mind on this debate

but clearly does have some concerns. These include: how it will track the results of its

funding in such alliances; how they contribute to a more independent and diverse civil

society in the South; and what the added-value is of channelling funds through a Northern

rather than a Southern affiliate.

The strategic positioning of AADK in recent years has been a good ‘fit’ with a number of the

strategic goals of the Civil Society Strategy. In particular:

Page 19: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

11

The growth in its media and campaigning work, in association with its coordination of

the Activista programme, helps to “promote a vibrant and open debate on poverty

reduction” (p 7).

Its programmes of capacity development support - for example, through T4C and

P4C - contribute to promoting “a representative, legitimate and locally-based civil

society” (p 9) and “capacity development, advocacy work and networking

opportunities” (p 11).

Its strategic focus on governance and, in particular, on support to rights-holders to

hold duty-bearers to account is completely aligned with the “promotion of focus on

rights” (p 12)

Its P4C and Global Change programmes play “an important role in Danish

Development cooperation … by injecting a people-to-people dimension….that

creates a greater understanding and support in Denmark”.(p 16)”

Danida is due to revise its strategy in 2012. The TOR for the pre-study of the evaluation of

the current strategy suggests that many of the trends outlined above are liable to continue or

even strengthen. In addition, the TOR suggests that different modalities for supporting

southern civil society are being explored. These include pooled/joint funding arrangements;

direct support to the Southern civil society versus support through Northern NGOs; and

promotion of diverse versus focused civil society support i.e. looking for new types of

partnerships. It is not unreasonable, therefore, to anticipate the following:

A much stronger focus and requirement from Danida on the demonstration of results.

Strong emphasis on the need to focus and to show professional expertise and value

added both to partners in the South but also to Danida.

Support for programmes which clearly show ownership and leadership from the

South including more funds being channelled directly to the South

An expansion of the number organisations covered by the framework agreement

within Denmark, and a possible reduction in the total funds available for those

agreements versus other modalities (with the former being more likely than the

latter).

Continuing focus on poorest countries and so called ‘fragile states’.

Interest in new partnerships e.g. with private sector actors or with non-NGO parts of

civil society in the South.

Continuing questions around the role of Danish participants in international alliances.

A possible interest in diaspora communities and the opportunities they offer for

connecting development internationally and domestic agendas.

AADK is reasonably well-positioned with regard to this scenario. Its programme expertise in

governance and youth is in line with current and emerging donor interest. This is evidenced

by the additional Danida funding that AADK received for work with young people in the

MENA region. The emphasis on capacity development is also compatible with emphasis on

southern ownership. The focus on youth also offers potential opportunities with other

collaborators - for example, there is interest from other international NGOs in the training for

young people offered by the Global Platforms. The growth in the application of new

technologies to establish closer, more immediate global connections offers opportunities for

new, different solidarity relationships outside the aid chain. AADK’s continuing emphasis on

people-to-people connections puts it in a good place to explore what these might look like.

However, the role of internationalisation and global NGO alliances will continue to be under

scrutiny. We believe AADK will need to continue to demonstrate how it contributes to

strengthening independent civil society organisations in the South.

Page 20: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

12

If the above analysis proves to be correct, it may be prudent also to plan for some reduction

in Danida funding and the attachment of greater requirements or conditionality to this

funding. AADK should continue to develop and strengthen its management information

systems to be able to respond to the challenge of greater accountability in a way compatible

with the nature of its programmes and its own values.

3.2. Fundraising The introduction in 2006 of a requirement of 10% matching funding from Danish NGOs as a

condition of framework funding propelled AADK, along with some other Danish NGOs, to

initiate its public fundraising efforts. AADK was late into the fundraising market in Denmark

and started from a weak position. Denmark is a comparatively small, competitive market for

public fundraising with a less well-established culture of public giving than some other

European countries. Some Danish NGOs in receipt of Danida Frame agreements,

particularly those with a humanitarian mandate such the Danish Red Cross and

DanChurchAid, had established fundraising capacity and a marketable ‘offer’ to the Danish

public. AADK had neither. Nor did it have sufficient unrestricted income to significantly invest

in its own fundraising capacity. AADK has held, and still holds, a pre-eminent position in

relation to Danida Frame agreements, receiving approximately 40% more in grant funding

than the next biggest Danida partner, requiring a correspondingly bigger match funding

contribution.

AADK, therefore, has had to grow its public fundraising in recent years. While the public

fundraising market in Denmark has grown, it has also become even more competitive. There

are signs that this period of growth may have reached a plateau. AADK is competing in this

market with only limited resources in terms of investment capital and experienced

fundraising staff. In this context, AADK has set itself an ambitious target of raising an annual

income of 50 million DKK by 2017. In the short-term, it has also set a target of 77%

dependency on Danida funding by 2012.

AADK has already more than doubled its gross income from public fundraising from about 8

million DKK in 2008 to about 18 million DKK in 2011. This is an achievement, albeit from a

very low level. However, net income has declined slightly in the last two years, largely due to

a higher level of investment. Net income is currently about half of gross income at about 9

million DKK. AADK has been able to meet its 10% matching funding requirement (currently

at approximately 13 million DKK) by supplementing its net income from public fundraising

with other income streams. The evaluation report on fundraising provides a detailed analysis

of the performance of AADK’s main income streams, none of which has demonstrated

significant net growth during this period.

AADK currently has a committed, enthusiastic fundraising team whose performance has

been hampered by staff turnover and overstretched capacity. Limited resources and human

capacity and pressure to achieve results have meant that insufficient investment has been

made in developing good donor intelligence and new fundraising products. Most importantly

the activities and priorities of the team have not been informed by a well-researched

fundraising strategy. A fundraising strategy paper in Danish made available to the evaluation

lacked sufficient detail. An earlier, detailed AADK fundraising strategy for 2009-11 developed

with the support of expertise in AAI was not known to the fundraising team. Although current

fundraising activities mostly correspond to the actions planned in the strategy, it would have

been preferable if the strategy had been used explicitly by the team to guide and monitor

fundraising activities.

Page 21: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

13

AADK will need to be realistic in setting fundraising targets in what may be a stagnating and

competitive market. It is unlikely on current trends that AADK will meet the targets it has set

for 2012 and 2017. To achieve its fundraising targets AADK must gain access to a wider

audience and provide a clear and coherent connection between values, messages and

products. For example, a new AADK strategy will need to identify how it will manage its

identity in Denmark so that it can combine a ‘youth activist’ profile in campaigning with the

imperative to increase its unrestricted income from the general public. As one respondent

said, AADK is currently in dialogue with the young while asking for financial support from the

old. Two broad approaches to future fundraising have been discussed. These are not

incompatible but will require sensitive management to clearly identify target audiences and

new fundraising products appropriate to these distinct audiences:

Invest in new approaches e.g. electronic platforms, to raising funds from or through

young people who are attracted to AADK’s core messages.

Increase and renew a dialogue with ‘mature’ individual donors building on classic

instruments such as membership, product-oriented campaigns and legacies.

AADK may consider supplementing its fundraising and communications with a strategy for

making full use of social media, first by assessing the availability and potential of new social

media products. Online fundraising is growing at a pace. There are a range of small to large-

scale opportunities which, if adapted to the Danish context, might help to both raise funds

and awareness of AADK among the Danish public. These include ‘widgets’ such as ChipIn

to enable individuals to raise small sums; fundraising through mobile platforms, for example,

through games with contributions per download; and online communities such as Care2

which encourage social networking in support of ‘good causes’.

Most Northern NGO public fundraising strategies are based on the demographic realities of

public giving - that peak and regular donations are to be found among the older segments of

the population. Whatever strategy is adopted will require a commitment to mainstream

fundraising in AADK organisational culture. It will also require a higher level of investment.

AADK has benefited in the past from fundraising support from AAI and is currently seeking

additional support. A revision of the conditions surrounding Danida Frame funding could

potentially provide the key to increased investment in unrestricted income.

The present stipulation that Danida partners raise 10% match funding from their own

fundraising towards the programmes supported in the Frame agreement puts AADK in a

‘Catch 22’ situation. To raise its income from public fundraising it needs to invest but it has

no surplus unrestricted income to invest since it is allocated to programmes. The rationale

for a match funding requirement must be that Danida partners actively seek to reduce their

dependency on Frame funding. This objective would be better served in AADK if it were able

to use its 10% match funding flexibly. This would enable it to invest in programme activities

not covered by the Frame Agreement including its own marketing and fundraising activities,

thereby enabling it to reduce dependence on Danida funding.

For the immediate future the evaluation recommends AADK consolidates its fundraising

efforts and becomes better at doing what it already does. Some activities that would

strengthen the performance of core funding streams include:

Develop a coherent fundraising strategy - with set of objectives, income streams

and key performance indicators - that matches its strategic strengths.

Invest in detailed intelligence on target audiences and relevant products.

Page 22: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

14

Invest in training staff with no previous direct fundraising experience.

Assess the business case of emergency fundraising and its effect on AADK

brand.

Assess the business case to build capacity to access and manage EC funding.

Increase the level of awareness and understanding of public fundraising in Board,

management and staff.

Close the national collection by 2012 and reallocate resources to building up new

income streams from social media or strengthening the institutional fundraising.

Clarify the desired, specific spin-off effects of the case story on the TV Gala and

use this as a lever for approaches in the other income streams. Make optimal use

of the Gala as a branding opportunity through involving volunteers and in support

events.

Shape more events for fundraising through better planning rather than adding

fundraising to existing events.

Clarify how domestic ‘political’ profile can be reconciled with the approach to

fundraising from the corporate or business sector.

Identify a media partner to match to Women for Influence members and the

target group they provide access to (business-oriented media or a magazine with

a more high society or ‘intellectual’ profile).

Consider increased investment in targeting foundations with tailor-made

applications.

Develop of a systematic donor relations plan and share it in the organisation.

Fast-track the resolution of barriers to in utilising the database/ customer relations

management (CRM) system at its full potential.

Develop a strategy for raising funds and awareness through social media.

3.3. Media and campaigning Campaigning has had a relatively short history within AADK and currently organised in

association with the recruitment, retention and training of a large number of volunteers.

Drawing upon this tradition of volunteer activism, AADK is committed to building a campaign

organisation in Denmark based on youth activism to “create concrete change for poor

people”.

The Activista programme has contributed positively to AADK’s domestic profile as a youth

activist organisation. There is evidence that AADK provides the most attractive ‘offer’ and

facilities in Denmark for young volunteers who want to be activists in a social, global cause.

However, only few volunteers become very involved as core Activistas and there is a high

turnover of these as they tend to be over-utilised by campaigns staff.

AADK campaigns are currently carried out by a small team on a limited budget.

Nonetheless, the team has been able to establish a good campaigning profile in Denmark

and has strong, positive relationships with other Danish NGOs and Danish journalists. It has

also established some good internal practices (including the “prouds and sorries” process for

institutional learning and establishing weekly meetings to coordinate policy and media work).

AADK’s press, media and public relations work provides strong media coverage and has

helped to ensure that AADK is a credible, authoritative voice in the Danish press. The

evaluation suggests that AADK’s media work performs above its level of resourcing. External

stakeholders rated the quality of AADK’s press and campaign materials very highly.

There is evidence of AADK’s ability to respond quickly on relevant issues in the Danish

development debate although a number of interviewees commented that the emphasis was

Page 23: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

15

sometimes on visibility rather than influence and impact. It is worth noting that AADK’s

strong media performance has not significantly increased its public recognition. AADK

featured as number 26 of all Danish NGOs in a 2009 survey, quite a bit lower than a number

of smaller organisations.

AADK has been active on a number of campaigning issues in recent years, for example:

Influencing Danish development cooperation policy.

Climate debt activism at COP+15.

Advocacy on the reform of EU’s Communal Agricultural Policy.

Advocating for a Financial Transaction Tax, the ‘Robin Hood tax’.

Campaigning on a halt to the production of non-sustainable biofuels.

Promotion of Fair Trade at the Fair Fælled Festival.

There have been a limited number of campaign evaluations so it is hard to assess the level

of impact of campaigns with any confidence. However, some notable successes are

apparent. For example, AADK’s policy work on biofuels - including the public launch of a

report, public debates and other activities - gained much attention. This attracted the interest

of the press, opinion makers and decision makers and stimulated wider public debate. By

2011, AADK had established a reputation in Denmark as the NGO to go to for analysis on

the impact of biofuels on food security.

Overall the evaluation found that campaigns and policy materials, briefing materials and on-

line campaigning offers were of good quality.

AADK’s commitment to campaigning, coupled with the opportunity presented by its

relationship with AAI, offer great scope for AADK’s campaigning work to become more

effective in the future. To date, there has been a continuing tension between two competing

views of campaigning:

A ‘popular mobilisation’ approach based on youth activism to raise public awareness

of development issues and the visibility of the organisation;

A focus on influencing policy and practice to achieve ‘concrete changes for poor

people’ in the South.

While both approaches are not incompatible, AADK must resolve which is primary in order to

achieve its ambition of becoming an effective campaigning organisation.

The emphasis on youth activism has predominated to date. The history and traditions of

AADK in ‘connecting people’ have contributed to a focus on popular activism to raise public

awareness on development issues. This is reinforced by its relationship with Danida who for

decades funded AADK as one of the principal suppliers of development education in

Denmark. This work now constitutes only 2% of AADK’s funding from Danida. If AADK is

going to reconsider the scale and approach of its campaigning it will most likely need to fund

this outside of the Frame Agreement with Danida.

Nonetheless, the evaluation believes that AADK should redefine its vision of campaigning

and policy advocacy, and move incrementally towards prioritising policy and practice change

for the benefit of poor people in the South as the ‘end’ purpose to which mobilising activism

is one ‘means’. This would require AADK to take a more strategic approach to campaigning

i.e. to identify an issue and plan to fully engage the organisation for a sustained period to

achieve ‘concrete changes for poor people’. There has been a tendency to work on too

Page 24: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

16

many issues for too limited a period, and to react to opportunities for visibility rather than

influence and impact.

The evaluation recommends a number of steps to enable AADK to increase its overall

effectiveness as a campaigning organisation. Key among these is the need to establish

planning processes and mechanisms to enable it to prioritise its campaigning and “do less,

do better” i.e.

Involve other departments e.g. International Programmes and Fundraising, in

planning campaigns. Strengthen internal communications between Campaigns

and the whole organisation is working towards the same objectives.

The Campaigns team should focus on campaign planning and implementation.

Volunteer management should be a separate function.

Prioritise AADK campaigns on explicit criteria such as the potential for change,

the scale of impact, the evidence available.

Base campaign planning and strategies on evidence; a theory of change; the

likelihood of achieving desired policy changes; and criteria for when and how to

mobilise supporters as well as other advocacy approaches. Tools like power

analysis and stakeholder analysis can help with the development of soundly

based campaign plans.

AADK campaigns need to make a link between the campaigning propositions and

AAI’s Southern programme experience to support their legitimacy and

accountability. Campaigning propositions need to be substantiated by robust

evidence.

Monitoring and evaluation of campaigning work needs to move beyond

monitoring activities and outputs to considering whether advocacy ‘outcomes’ are

being achieved.

3.4. Global Contact

MS was founded on the belief in the power of people-to-people initiatives to create the

foundations for peace and social justice. AADK’s identity is still significantly shaped by this

early aspiration, resulting in an emphasis on the value of individual and person interaction

and the benefits they produce in terms of cultural understanding, learning and solidarity

between peoples. AADK’s global volunteering programme, Global Contact, was established

in 2006 with the mission of promoting these values along with a sense of ’global citizenship’.

Global Contact is included in the People4Change strategy although it is managed by the

Training4Change department.

Global Contact currently offers five different types of volunteer travel, including Young

Volunteers, Older Volunteers from 26+, Student Interns, Next Stop study tours, and Work

Camps. The largest group is Young Volunteers, consisting of one month training at Global

Platform and three months of placement in a developing country as well as a preparatory

and debriefing session in Copenhagen. This group is the main focus of the evaluation.

Since 2007, AADK has placed an average of 733 volunteers a year in 34 countries. The

largest number are placed in Africa, in particular in Kenya, Uganda and Tanzania.

The idea of Global volunteers is to create ‘potential change agents’ who are active in

promoting positive change on issues of development and social justice. The value of

placements lies less in the work the volunteers do, than the friendship, solidarity and

connections that are created. The aim is that young volunteers going abroad with Global

Page 25: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

17

Contact will become active in promoting social change after their stay, either with AADK or

other initiatives. The volunteers are told that they should not expect to change the world

during their stay, but can contribute to change the world after their stay.

AADK’s current objectives for Global Contact are to:

promote global citizenship and volunteerism through intercultural exchange;

build a strong volunteer and campaign network in Denmark;

strengthen the capacity of Global Contact as an income-generating business.

Overall volunteers display a high level of satisfaction with their stay abroad - around 80%

saying they were satisfied or very satisfied; and a similar percentage saying they would

recommend Global Contact. From the evaluation results, Global Contact achieves its goal of

supporting young people to gain a more global perspective fostering values of tolerance,

curiosity and intercultural exchange among volunteers.

In terms of the broader impact of the programme, it is difficult to offer firm conclusions about

the correlation between volunteers stay and greater engagement throughout their lives. The

data is insufficient because of the relatively short time between the placement and the

survey. Most respondents are from the 2009 cohort. There are some positive initial

indications, but AADK would need to have an improved data management system for

staying in touch with volunteers and following them systematically over several years post-

placement to draw more robust conclusions. Respondents to the surveys suggest there is

little impact on volunteers’ choice of study. There is some impact on their career ideas, but

only 16% register this as a strong influence. Much stronger is the impact on perceptions of

international development (72.7%) and global issues (62.7%). 74% of the total cohort

surveyed continue to be engaged with AADK as members, and approximately 50% as

activists, one year after their placements. This is positive but as membership is a

prerequisite for accessing the service it is not clear whether this is an active choice or how

long it continues. The percentage of those indicating that they do volunteer work now is

approximately the same as those indicating they were active prior to travelling with GC.

Interviews with AADK staff, however, suggest that a large percentage of activists who work

with AADK are former Global Contact participants

Host organisations are very positive about the benefits they get from volunteers (although

volunteers are more ambiguous about the benefits they bring). These range from simply

having an extra pair of hands to more concrete outputs such as building a website or

fundraising. Having someone from another culture working with them can bring new

methods and ideas and can give energy, enthusiasm and momentum to work. Host families

are also very appreciative of having volunteers staying with them. They clearly bring some

economic benefit, but families also point to broader benefits (particularly for children) such

as learning and practicing English, bringing a more global outlook and providing insight into

the diversity of peoples and cultures.

There is also evidence of tangible benefits to AADK - volunteers become members and in

some cases financial supporters. Global Contact contributes approximately 3 million DDKK

to the Global Platforms (see Section 3.3.) through course fees.

Global Contact could improve its financial and market information systems to support better

business modelling and strategic decision-making. For example, the programme is

described as recovering all its costs without taking into account Danida funding for Global

Volunteer trainers.

Page 26: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

18

The evaluation was not able to establish independently how robust the market for volunteers

is in Denmark. A more thorough market analysis and client information would be needed for

this. Price is an important factor. For example, increases in the visa price in Tanzania have

already had an effect on the number of volunteers. It is not clear to what extent investment in

and marketing of the quality of the offer could sustain the current price and even support a

slight increase (needed for full cost recovery).

The overall satisfaction ratings suggest that the quality of the service that Global Contact is

offering is good and meeting client’s needs. The qualitative comments, however, from

volunteers do suggest that there are areas that could be improved if the programme is to be

run more in conjunction with AAI country programmes. In particular, it is important that

comparatively unskilled, short term volunteers are provided with meaningful work and

concrete tasks.

There is a lack of knowledge about Global Contact among AA country directors and no

documented discussion of it between AADK and AAI at a senior level. This lack of visibility

may reflect some sensitivity about the programme from both sides. Some resistance in AAI

may be caused by a perception of the programme as being anachronistic and benefiting

young Danes rather than playing a useful developmental role for hosts or communities. This

would be unjust to the programme as there appears to be real mutual benefit and indeed

enjoyment in the exchange in a way that could be described as a ‘win-win’. Trends suggest

that there is a real appetite for more direct unmediated connections between people. Far

from being anachronistic, the programme could be considered to contribute to the long-term

development of global social capital and thus have relevance to AAI’s commitment to

support global movements for change.

AADK needs to dialogue with AAI about the future role of Global Contact. A few AA country

offices have made some progress in integrating Global Contact within the country

programme. However, the future of the programme in AAI revolves around its identity as a

business or a social enterprise. As a ’business’ it will need to offer a benefit, in terms of

human or financial resources, to country offices that manage it. As a ’people-to-people’

programme within the new AAI strategy, it would need to extend its focus beyond Danish

youth.

The route to greater alignment could be through the placement of volunteers with AA

partners. Matching partner work with the mainly unskilled and short term nature of the

volunteers would be challenging, although as AAI expands youth work this might offer some

opportunities.

The programme has already begun to look at ways that it could expand the model to middle-

income countries where there is a young, middle class population who could manage to

raise the funds to self finance. Moving quickly with a pilot on this could be useful evidence to

test the wider potential of the programme and to expand it to a more South-South focus.

Another option could be to seek grant funding to enable more youths from marginalised

communities to benefit from this exchange.

The evaluation recommends that AADK continues to develop Global Contact as a way to

attract young people to global volunteerism and activism and that it should:

Page 27: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

19

Review with AADK and AAI different models of implementation at a country level

and the learning from and impact of the programme in countries where

integration is more advanced.

Encourage experimentation - for example, consider expanding internships, 26+

volunteers; look for partnerships with other organisations/businesses that may

have skills that match partner demands. This might have the added advantage of

encouraging more collaboration from the Danish private sector in development

initiatives.

Retain Global Contact Coordinators in all focus countries where there are large

numbers of volunteers and where greater integration with AAI is being pursued.

Consider reducing the number of countries offered to volunteers. This might have

an impact on the attractiveness of GC to volunteers but would be offset by

investment in ensuring a quality experience and the ability to review integration

models with AAI.

Consider piloting the programme with volunteers from a Southern or BRIC

country e.g. MENA region, Latin America or India.

Continue to focus marketing on the uniqueness identified in the strategy, with a

stronger emphasis on GC being part of a world-wide organisation fighting poverty

and an invitation to stay involved after the volunteer experience.

Consider employing a full-time professional marketing expert with responsibility

for market analysis and marketing in order to increase Global Contact’s market

share.

Revise financial and management information systems to improve business

modelling and decision-making - for example, by accounting for hidden subsidies

and costs and providing consolidated income and expenditure accounts that

would allow better comparison across countries of the costs of the programme

and different models.

Improve the post-placement satisfaction survey format and complement it with an

ex-post survey after 3-5 years to help build client intelligence and contribute to

tracking impact.

Introduce an annual or bi-annual survey to increase the focus on the satisfaction

and outcome for the host organisations and families and ensure their suggestions

are regularly monitored.

4. AADK in ActionAid International

AADK negotiated four ‘side-agreements’ as part of its affiliation with AAI in 2009. These

represented a set of exceptional arrangements for AAI. MoUs were signed for AADK to

continue to implement two of its key programming areas - T4C and P4C - over the next two

years in coordination with AAI. It was agreed to review the experiences of these approaches

before the end of 2011 to make an informed decision about their potential to be

mainstreamed within AAI. In the meantime they were to be considered ‘parallel’ programmes

to country programmes - and the AAI Secretariat - with separate budgets. It was anticipated

that new management arrangements would be put in place if they become part of AAI

strategy.

At the same time, AADK assumed responsibility for leading two international areas of work

within AAI. In line with its own chosen area of competence, AADK took responsibility for

promoting and supporting the Just and Democratic Governance (JDG) theme, one of five

priority themes within the AAI 2007-11 strategy. The only other example of a member

affiliate leading a strategy theme/priority at the time was AAUK’s lead responsibility for

Page 28: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

20

humanitarian response. AADK also agreed to take responsibility for taking forward the

Activista initiative across the federation. In both cases, the agreement was to be reviewed

after two years and to identify any lessons to be learned with regard to models of delegated

leadership to a Member affiliate.

In this section we summarise how the evaluations have assessed the relevance and

effectiveness of each of these programming areas and offer some conclusions and

recommendations as to their potential contribution to the AAI strategy. We will also look what

lessons, if any, the ‘pilot’ experiences of AADK leadership of the JDG theme and Activista

programme offer for the delegation of international work to Member affiliates.

The analysis reveals a good track record of achievements in all four programme areas. Each

evaluation independently recommends that AADK, on the basis of good performance to

date, continue to provide leadership within AAI to the programmes under review. However,

two underlying themes also emerge:

The need for a ‘unifying framework’ e.g. theory of change, capacity building

framework, to enable programmes to identify target audiences, appropriate

products and potential synergies with other programmes.

The need to clarify its approach to Southern civil society and target and report

more explicitly on services for AA country offices, partners and local CSO staff.

4.1. Leadership of the Just and Democratic Governance theme

AADK and AAI entered into a two-year agreement in December 2009 for AADK to lead the

Just and Democratic Governance (JDG) theme across the federation. AADK assumed

responsibility for a small International Governance Team (IGT) early in 2010 which by

November 2010 had been expanded to a team of ten.

Despite a delayed start, the work of the IGT is viewed very positively by AA country

programme staff who have had contact with it. 87.5% of Country Directors and 91.6% of

governance focal persons who responded thought the work of the IGT ‘moderately’ or ‘very

important’ to helping deliver their governance work. A similar percentage would welcome

AADK’s promotion of and support to governance work in the new strategy.

Perhaps the most significant achievement has been the training of more than a thousand

ActionAid, partner and other CSO staff in 53 governance courses in 2010/11. Both

participants and commissioning managers in survey returns expressed a high level of

satisfaction with the quality and effectiveness of these courses, and reported that the training

had improved the delivery of governance programmes. This has created an important,

skilled constituency of support for governance work under the new AAI strategy.

The quality of manuals, discussion papers and handbooks produced has also contributed to

a positive view of the work of IGT. 87.5% of Country Directors and 100% of the governance

focal people who responded rated IGT manuals as ‘good’ or ‘excellent’. The Economic

Literacy and Budget Accountability in Governance (ELBAG) handbook series is the key,

user-friendly resource for ActionAid and partner staff. Two handbooks have been produced

to date; the remaining three are due to be published and distributed before March 2012.

A 5 million DKK Seed Fund established in 2010 to supplement or start up governance

programmes at country level has also played a major role in publicising and popularising the

work of the IGT. So far 32 projects have been funded in 17 countries. In the case of

Page 29: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

21

Myanmar, it enabled pilot work on local government accountability to leverage EC funding to

scale up the work.

IGT has also been responsible for the delivery of the AAI priority National Development

Strategy (NDS) project, focusing on the development and promotion of alternative models of

development. IGT’s work on this priority theme (involving stakeholder mapping,

commissioning research and concept papers, and convening meetings) is rated positively by

country directors and governance focal persons and will form the basis for national advocacy

work in 2012.

The IGT has been unable to meet its targets in some other areas. The plan to map

governance work and make it available on a searchable database on the AAI intranet is

unfulfilled as yet. An assessment of the capacity development needs of governance focal

persons has been postponed. A number of concept papers have been delayed or

postponed. Plans to incorporate Advisors and Inspirators into governance programmes have

yet to be fulfilled. Success with support to fundraising efforts has been limited to date.

Efforts to promote the sharing of learning through exchange visits and web-based

applications have been limited. Having a team leader who was well-connected within AAI has helped the IGT make progress

despite initial delays. Adequate planning, reporting systems and arrangements for

teleconferencing are in place, but inconsistently implemented. A tighter planning framework

and more effective prioritisation would have helped deliver a more achievable work plan.

None of the management arrangements put in place under the MOU to ensure that AADK’s

leadership of the JDG theme was accountable to the AAI Secretariat, have been

implemented. There is neither evidence that this was intentional nor that it has had a

negative effect on the work of the IGT. However, systems of formal accountability to the

Secretariat have been weak.

A number of lessons emerged from the experience of the IGT to date:

It will be important not to repeat in the new AAI strategy the break in continuity

and loss of institutional memory that contributed to delays in taking the work of

the IGT forward.

The delegation of international work to Member affiliates in the new strategy must

be accompanied by a clear mandate and project structure, strong leadership,

clear TOR and a transparent system of accountability to the Secretariat.

Support to governance work under the new strategy should exploit more fully the

opportunities to communicate effectively using the wide range of options

available with electronic media. The work of IGT has been knowledge-rich, but

mostly confined to the provision of training and written outputs. With the

establishment of a sizeable constituency of AA and partner staff who have

benefited from these inputs, greater attention should be paid to horizontal system

of learning including exchange visits, easy-to-use content through the web, user-

friendly database/s and virtual communities of practice.

Future support for international work on governance work would benefit from a

more explicit capacity building framework within AAI. This should encompass

training; institutional strengthening; mentoring; knowledge sharing including peer-

to-peer exchange and web-based tools. This would enable AADK to coordinate

and achieve more synergy between its different programme components.

Page 30: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

22

Such a framework should clarify whose capacity in governance work is to be

strengthened so that it can customise delivery systems and materials for them.

AADK must be able to better demonstrate how and how much its support to

international governance work within AAI provides direct benefits to Southern civil

society. This will require, for example, T4C/TCDC to revise its data management

systems so that information on participants in governance training can be

disaggregated.

The new AAI strategy offers an opportunity for AADK to continue to promote and support

governance work in the federation by including a specific objective on governance and

recognising the delegation of international work to Member affiliates. This evaluation

concludes that the credibility, experience and expertise accumulated under AADK leadership

of the governance theme are assets which should continue to inform and support

governance work within the new strategy.

It is likely that working groups will be established to support the delivery of objectives and

key promises in the new strategy. A key dimension of choice will be whether these groups

will be supported by a budget and the level of leadership responsibility delegated to them.

This could result in different approaches ranging from a governance ‘community of practice’

facilitating learning across programmes to a governance ‘team’ coordinating national and

international work to achieve Key Promise 3.

The latter model would be best suited to enable AADK to continue offering leadership and

funding to governance in AAI through its proposed model of programme and seed funding,

and support for ‘knowledge hubs’ in the region.

In either case, AADK must meet the expectations of both AAI and Danida regarding

accountability. In the former case, the Chief Executive and/or Director of Programmes must

have responsibility for how best to leverage AADK funding and expertise in support of

Objective 2. In the latter case, AADK must be able to demonstrate its added value as a

conduit for Danida funding to the South and the contribution that this funding makes to

strengthening Southern civil society.

4.2. The Activista programme

Activista was established in 2007 as a global movement “for young people who want to work

together for a better world without poverty.” In 2009 AADK took over the coordination of the

Activista programme from Action Aid UK and the Campaigns Department in ActionAid

international. Since then it has focused on providing young people with skills to become

‘change-makers’ or ‘drivers for change’.

AADK’s support to the training of young activists through the Global Platforms is sustained

through Danida funding. The Training4Change evaluation confirms that respondents report

the training to be of high quality and that it has increased their ability to campaign and

provide local leadership for change. Stakeholders have pointed to the participatory approach

to training and the action-oriented ‘learning by doing’ approach as particularly valuable.

The challenge of linking up Activistas with AAI global campaigning initiatives in Denmark and

across the world is more complex. A number of respondents felt that global campaign

themes such as Hunger Free could be useful for making people feel part of a global

movement but functioned less well as drivers for activism at local level. This would seem to

hold true in the North and the South. ‘Top down’ campaign coordination can be perceived by

Page 31: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

23

young activists as treating them only as “campaign soldiers” to enable ActionAid to get

numbers on the street.

As in the case of AADK campaigning, the evaluation raises the issue of the theory of change

that underpins the role of the Activista programme. Is the end purpose to mobilise youth to

communicate a message through an activity? Or is it part of a more comprehensive

advocacy strategy to influence policy and practice? Section 5 of the new AAI strategy on

‘Our theory of change’ - which suggests a model of change led by the South and supported

by solidarity, credible rights-based alternatives and campaigns that address the structural

causes of poverty - does not provide a clear answer to this.

The human and financial resources allocated to the programme were, and are, inadequate in

light of the scale of ambition of its long-term goals. The work associated with the transfer of

the Activista coordination from AAI to AADK was underestimated on both sides.

Nonetheless, respondents recognise that AADK taking over Activista not only rescued the

programme, but has taken it forward to a new level of achievement. Country programmes

collaborating with Activista view the overall administration of the programme as satisfactory,

and that it has been relatively good at responding to their requests.

Most respondents felt that the physical location of the Activista coordination was not a key

issue at present but that the lines of accountability to the AAI Secretariat should be much

more transparent and better defined. When Activista is absorbed by the Secretariat some

thought it would be more appropriate to locate its coordination in the South - to be closer to

the priorities of rights holders and target groups. Relocating the coordination would help to

ensure that the accountability of Activista and its links to the Action Aid International

campaigns team.

The new AAI strategy clearly commits the federation to supporting youth activism globally.

Objective 3 commits AAI to supporting youth as ‘powerful drivers of change’ and Key

Promise 6 commits AAI to mobilising over five million youth towards building a poverty free

planet. Section 6 on ‘Our approach to bringing about change’ commits AAI working more

with young people as ‘powerful drivers of change’ and to investing in their capacity to

organise and run campaigns, promoting their leadership and linking them to dynamic

movements for change.

It is not clear what role the Activista programme will play in achieving Key Promise 3. The

evaluation found some difficulty in comprehending when a youth activist is or is not an

Activista. Certainly the numbers of youth who participate in Global Change courses through

the Global Platforms is minute compared to the scale of ambition in Key Promise 6.

To ensure that Activista contributes effectively to the commitments in the new AAI strategy,

the evaluation recommends that:

AADK continue to build on its achievements to date and continues to coordinate

Activista within AAI. Consideration should be during the next strategy period to

relocating and reassigning its international coordination to a Southern office.

Clarify the role of Activista in contributing to policy and practice change,

specifically to the achievement of Key Promise 6. Define more accurately the

concept of Activista both for internal purposes of management, monitoring and

reporting, but also for easier external communication.

Page 32: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

24

Confirm the level of financial and human resourcing available from AAI to support

the Activista programme. Consider how to diversify the funding base for Activista

through specialised fundraising products targeted at youth.

Develop action plans to enable Activistas to contribute to Objective 4 and Key

Promise 6 with objectives and targets that are achievable in line with the

resources available.

Communicate the Activista programme to country programmes and ensure it is

fully integrated and ‘owned’ at country level through binding agreements.

Make campaign themes flexible, and the product of dialogue with youth itself, so

that youth activists in each country can adapt to issues relevant and motivating

for them.

Facilitate cross-regional experience sharing among Activista coordinators to learn

and build on experience.

AADK and AAI work together to devise a M&E system for the programme that

gathers appropriate data on youth activism at country level in association with

reporting on Key Promise 6, including details of Activista numbers, activities and

attitudes.

4.3. Training4Change

The TrainingforChange (T4C) programme was formed in 2008 and consists of the long-

established training centre in Tanzania, TCDC, and seven small-scale training centres or

‘Global Platforms’ based in Tanzania, Kenya, Nepal, El Salvador, Denmark, Jordan. A

Global Platform is about to open in Myanmar. T4C has achieved a great deal in a short

period of time, in terms of navigating through a major process of change, embarking on an

ambitious growth strategy and providing good quality training services to a range of

stakeholders. The scale of operations of T4C is impressive. Since its inception it has

delivered successful training courses to over 5,000 participants at TCDC and around 2,500

at the Global Platforms.

Since 2009, T4C has delivered a suite of governance courses tailored to the needs of AAI,

alongside its general skills courses. AAI accounts for 30% of participants on TCDC courses.

We have seen in Section 3.1 the high level of satisfaction expressed by stakeholders with

the relevance and quality of the governance courses delivered by TCDC. Moreover, a high

percentage report that the learning from the course has been useful in their organisations

and contributed to their delivery of governance work. The most frequent comment on the

quality of the courses was a concern to base the training more on practical experience. T4C

has developed a number of MOUs with AA country offices to deliver a range of courses and

plans to develop more.

Global Platforms deliver a variety of preparatory training courses e.g. to Global Contact

volunteers; Global Change courses on campaigning for Danish and Southern youth; Training

of Trainers (TOT) training; and a suite of ‘Global Toolbox’ courses focusing on Global

Citizenship. The majority (68%) of participants on Global Platform courses are Danish youth.

The participation of Southern civil society youth is currently low but T4C have had some

success in targeting new courses such as Global Change and Global Toolbox, to the needs

of local CSOs, youth groups, and individuals.

Participants generally rate Global Platform (GP) courses positively in terms of relevance but

the picture is more mixed than training in governance. TOT clearly has a strong impact with

85% of participants saying the course contributed to how well they now train; 93% saying it

had improved their career opportunities; and 97% saying they would recommend it. Only

Page 33: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

25

53% of Global Contact volunteers rate their course as ‘relevant’ or ‘very relevant’. The

evidence is that the course helps the volunteer understand global issues and live in a

developing country more than in preparing them to live with a host family or work with an

organisation. There is evidence the Global Change course is having a positive impact, being

rated 7.5 on a scale of 1-10 by participants. However, 20% rated it ‘5’ indicating a need to

better assess the needs of participants.

T4C intend to improve TOT, Global Change and Global Toolbox courses over the coming

year. The evaluation found that the Global Toolbox course had a mixed response and

recommends that the courses be reviewed and possibly repackaged and rebranded. There

is room for improvement in other areas. For example:

TCDC and GPs invest insufficient time providing feedback to trainers to ensure

continuing improvement in the delivery of courses. At TCDC, for example,

although the understanding of participatory approaches is high, the ability to

deliver participatory training is not consistent across all trainers.

T4C currently reports to Danida on the basis the number of training days. The

evaluation recommends the use of indicators that provide a better picture of the

scope and effectiveness of T4Cs work - for example, number of courses, number

of participants, participant profiles, average numbers per course, average

evaluation scores, and total number of course days. T4C should also increase the

degree to which it assesses the impact of courses by tracking learning, behaviour

and results of courses as much as possible. The new database may support this

but further work will need to be done to ensure that reporting is given greater

priority and integrated into the ways of working.

T4C reports that it is on course to meet its income and ‘surplus’ targets. In reviewing the T4C

‘business model’ the evaluation found it difficult to analyse the financial interdependency of

different business units; their overall direct and indirect dependency on Danida funding; and

what, therefore, constitutes a surplus or profit. TCDC is also in an unusual financial position

in that the building, land tax and property tax are currently not charged for by the Tanzanian

government. About 46% of T4C income comes from Danida funding and the aim is to reduce

it further. T4C would currently be unable to provide the current range of training courses at a

cost recovery level without the subsidy of Danida funds. GPs are heavily reliant on income

from Global Contact - from the fee income from volunteers and Danida funds to pay the

trainers. T4C is actively exploring alternative sources of income and has had some success

in obtaining new grant funding and building new partnerships. Nonetheless, the current level

of dependency on Danida funding means that T4C is vulnerable to changes in donor funding

policy.

Both GPs and TCDC have strong selling points for their specific markets. GPs have a

participatory, active approach to learning well-suited to their focus on youth mobilisation.

TCDC also has participatory approach and has established a strong ‘brand’ reputation in

Africa. However, T4C is not without competitors, especially in other regions, and need to

invest more in understanding its market. Course costs for both GPs and TCDC are around

the average in their markets but are subsidised by Danida funding.

T4C will need to improve its marketing efforts and invest in marketing skills if it is going to

attract a bigger audience and improve its ability to generate surplus income. TCDC has a

new marketing strategy and plans to visit CSOs and AA offices across Africa. GPs need to

focus their future efforts on branding and packaging courses. The marketing of the GPs is

Page 34: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

26

currently inadequately resourced, being the responsibility of overworked camp managers

sometimes supported by interns.

T4C plans to expand its operations in the next few years through, for example, the

expansion in the numbers of Global Platforms. The evaluation recommends, however, that

T4C invest in consolidating the quality of its work before expanding further. A theory of

change should guide the growth of T4C. This would clarify who the target audiences are;

what their needs are; what products are best suited to meet these needs; how quality can be

assured; and what business model will sustain it over the next few years. Growing T4C

prematurely may distract it from focusing on these key improvements which should provide

the platform for future expansion.

“MS in Transition, September 2011” refers to gradually transferring the T4C ‘competency’ to

AAI in the longer term although AADK would like to retain a “substantial primary capacity” in

training. The evaluation recommends that T4C continues its collaborative relationship with

AAI, but continues to be managed by AADK during the next strategy period so that it can

build a sustainable business model based on its flexibility to respond to the needs of diverse

stakeholders.

The evaluation recommends a number of ways in T4C can continue to improve the

relevance and effectiveness of its ‘offer’. Key among these are that T4C:

Develop comprehensive strategy with clear objectives. This would identify

synergies across AADK; identify its target audiences and the plans to reach

these. This would clarify the relative importance of AA country-based staff;

partner staff and CSO staff as target audiences. As such it would help clarify

T4Cs role as a service provider e.g. to AAI or as an independent capacity-

building programme; when it should lead and when it should play a supporting

role.

Develop a management information system to gather data so that it records and

reports data on courses, activities and participant profiles by different participant

categories.

Invest in an M&E system to enable it to report on outcome rather than output

indicators. This will enable it to focus on the activities that have the greatest

impact.

Conduct needs assessments in the regions to determine which toolbox courses

or TOT related courses are demanded and whether they should be repackaged

and branded to clarify what is on offer and to whom.

Invest in the quality of the courses in terms of content and training

methodologies.

Increase consultancy and tailor-made work to improve the utilisation of the

facilities and trainers; monitor course ‘up take’ so that courses can be marketed

when ‘low uptake’ is observed.

Consolidate rather than expand Global Platforms to ensure they reach the right

audience with the right ‘products’. Invest in the quality of courses and their

marketing and branding.

Page 35: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

27

4.4. People4Change1

Overall, P4C represents a highly relevant contribution towards AAI’s goals and objectives,

with evidence of contributions towards the strengthening of the voice of rights holders and

subsequent impact on people’s lives (e.g. obtaining land rights; access to increased income

generating opportunities etc.). 73% of partners report (medium/high) increased impact in

their governance work.

The emphasis on a people-to-people, cross-border approach to capacity building is valued

by stakeholders and there is clear evidence that the P4C programme, has contributed to

partner capacity development in the relatively short period since its inception. For example,

survey results reveal that:

62% of partners said P4C represented ‘real and excellent value-added’ with no

significant difference in response between Advisors and Inspirators.

55% of partners indicated above average fulfilment of individual placement

objectives

58% of partners report (medium/high) improved internal organisational

functioning.

55% report (medium/high) increased relationships.

46% report feel that this is making a (medium/high) difference in peoples’ lives.

65% of partners reported medium/high increased capacities of individual

staff/members in the areas of new perspectives on governance, new knowledge,

enhanced skills and increased motivation levels.

Stakeholders refer to a number of value-added contributions of Inspirators and Advisors

including the:

Provision of external perspectives;

On-going accompaniment of processes;

‘Grounding’ of newly acquired skills or knowledge in concrete work activities; and

Advantages of being seen as a ‘neutral’ actor who can help overcome local

prejudices and speak their mind without being seen as having particular interests

at stake.

In most cases, P4C placements were relevant to the needs of Action Aid country offices and

their partners. 83% of all partner respondents indicated that the way they had been involved

in the original identification of the capacity building effort had been a positive factor in

generating the changes they achieved. The majority of Advisors and Inspirators indicated

that they knew of ActionAid Denmark and were motivated by the desire to work with the

organisation.

1 The evaluation was asked to review an example of the AAI Fellowship Programme as a possible

comparator with the P4C model. A review has been conducted but concludes that, with its focus on

community mobilization and empowerment, the Fellowship programme in Myanmar might be more

comparable with AAI ‘operational’ model e.g. in Local Rights Programmes. No further reference will

be made to the Fellowship programme in this report.

Page 36: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

28

The achievement of the P4C programmes is particularly impressive since it has taken place

at a time of significant change in the country programmes (especially the merger). The

Global Human Resources department in Copenhagen and P4C facilitators have played a

critical role in this respect. Efficient operationalisation of the P4C programme at the global

level has been a key contributing factor towards the achievements described above. Basic

programme instruments were designed and made available to the teams on the ground, and

the ‘learning approach’ enabled a process of constant review and adjustment as the

programme was unfolding.

There remain, however, some challenges to ensure that the P4C approach is fully integrated

with other AADK programmes and with AA country programmes; and to offer maximum

benefit as a tool to strengthen Southern civil society capacity. These challenges may be met

by:

A capacity development framework to identify who, why, how

A capacity development framework would clarify what is capacity and capacity building; why

the organisation engages in this work and how it fits with its overall theory of change and

overall programme strategy; whose capacity is being strengthened; what are the values and

principles which guide its approach; and how different methods are used. More specifically, it

would help P4C clarify the targets and methods appropriate to strengthen civil society

capacity, and to how to establish effective linkages with other programme initiatives.

Organisational priority 4 of the new AAI strategy refers to capacity building, but focuses more

on internal AA capacity building than partner capacity building as part of civil society

strengthening. AAI has invested substantially in developing and rolling out its programming

framework, the Human Rights Based Approach (HRBA), in recent years. The AAI Capacity

Building Initiative has produced a concept paper, attached as an annex to the HRBA

manual, which reveals a great deal of similarity with the thinking behind the P4C approach. If

formally adopted it would make P4C a highly relevant programme initiative, but at this stage

the paper does not seem to have been discussed widely. Currently however, there is no

other AAI reference document on partner capacity building that serves as a reference point

for the P4C programme or for national offices engaged in P4C initiatives.

A key question for a capacity development framework to clarify is ‘whose capacity?’ The

history of MS work in this field - and the rationale for Danida support - has been the

strengthening of Southern civil society partners. A contentious question is whether AA

country offices and local operational programmes - including those of Southern affiliates -

are considered to form part of Southern civil society. Currently AADK reporting does not

differentiate between AAI entities and autonomous CSOs as ‘partners’. The trend seems to

be to be to place Advisors and Inspirators with AAI entities. 65% of total Advisor placements

and 5% of total Inspirator placements have been partially or entirely with AA entities; 20% of

‘pipeline’ requests for Inspirators appear to be oriented primarily towards AAI initiatives.

Even when an Inspirator or Advisor is based in an AAI entity is rare that s/he is focused on

strengthening AA’s capacity alone. However, this is a sensitive issue since the perception of

local stakeholders can be that AAI is prioritising investment in itself at the cost of partners.

A capacity development framework would also help to clarify the roles played by Advisors

and Inspirators in partner organisations. Despite a highly positive picture, for example, there

remains an element of ‘gap-filling’ in the P4C programme, either because the request is for

someone to fill a position where no funding is available for a local staff member, or because

of a heavy emphasis on the ‘doing’ rather than the ‘advising’ or ‘facilitating’. This applies to

both Inspirator and Advisor placements with partners and AAI offices. In addition, there is

Page 37: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

29

evidence that the ‘inspirational’ role of Inspirators, for example, is insufficiently understood

with a large number focused on supporting programme management systems and strategies

which, although relevant for many partners, is arguably is more ‘technical’ than ‘inspirational’

role.

Improve the profile and level of understanding of P4C within AAI

In its inception the new P4C programme emphasised the people-to-people approach but the

capacity building dimension has emerged more strongly as the programme has developed.

The people-to-people approach, particularly the South/South dimension, resonates with the

AAI strategic emphasis on people taking action together, but there continues to be

misconceptions among AA country office and partner staff about the P4C approach. Some of

the difficulties described above have their origins in these misconceptions and there is still a

residual profile of P4C as being about ‘personnel sending’ not programming. The hosting of

P4C in ‘Global Human Resources’, for example exacerbates this.

Both AAI and AADK share responsibility for such misconceptions. AAI was committed to

assign senior staff member to follow the P4C process and coordinate among AAI

stakeholders. This has not happened to any great effect and opportunities to dialogue and

build up understanding about the P4C approach within AAI, therefore, have been limited. At

the same time AADK opted for a ‘softly softly’ approach to incorporating P4C within AAI,

being wary of being seen to ‘push agendas’ as a recent Northern affiliate. Rather than draw

attention to P4C within the Federation, it chose to build understanding from the ‘bottom up’

as country teams became exposed to the programme’s contribution and potential. This may

well have been a sensible tactic but it has also involved some opportunity costs to develop a

broader understanding of the aims, methods and benefits of the programme across country

programmes. The recent effort to produce informative fact sheets on P4C to be shared with

country offices and posted on the AA intranet (HIVE) is a welcome but overdue initiative,

Manage P4C for increased effectiveness

There are some challenges in programme management which will require attention in the

next period. These include the need to:

Better align P4C decisions and implementation with in-country programme planning;

Adapt the model to respond flexibly to partner needs beyond the formal

categorisation of Advisors or Inspirators;

Develop a more systematic assessment process to ensure that placements are

demand-driven and offer an ‘enabling environment’ for the Advisor/Inspirator. New

AADK PME materials for governance programming incorporate a partner capacity

assessment tool but it is too early to say whether this will provide the quality of

assessment required;

Build the engagement of AAI country offices and ensure relevant staff fulfil their

responsibilities to the programme;

Strengthen the criteria on placements to avoid ‘gap-filling’;

Resolve outstanding logistical and operational issues e.g. regarding visa status and

medical insurance for Inspirators; in-country induction processes; the logistical and

administrative capacity of some country offices to facilitate Inspirator placements.

AADK, in MS in Transition 2011, outlines an intention to transfer the P4C ‘competency’ to

others e.g. AAI but to ensure a democracy focus within the programme. The evaluation

concludes that P4C has a valuable role to play within the new AAI strategy. The leadership

of P4C should remain a delegated responsibility of AADK for the next strategy period,

exercised in the service of AAI through a clearly defined mandate, MOU and set of

Page 38: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

30

management arrangements. It also recommends that P4C continues to work in support of

governance-related programmes and activities. There is not yet an adequate understanding

of the programme across AAI, nor the human resources at country level to ensure the

programme can be effectively implemented across all areas of work within the new strategy.

The next few years should ones of consolidation rather than expansion.

Key among the other recommendations are that:

AAI adopts a comprehensive capacity development framework that legitimates and

situates the P4C programme as a means of strengthening Southern civil society.

AADK clarifies the concept of partnership in relation to strengthening the capacity of

Southern civil society in line with the Danida Civil Society strategy.

In-country P4C facilitators should be retained where the HR capacity of a country

office is unable to absorb the relevant logistical/administrative responsibilities of the

programme. AAADK and AAI should agree a set of criteria and process to undertake

this joint analysis.

AADK planning and management systems are strengthened to ensure that P4Clinks

effectively with other programme areas such as T4C and support to governance work

to ensure maximum programme synergy.

AADK considers refining the P4C approach to offer a spectrum of people-to-people

capacity development approaches so as to be better able to respond flexibly to

partner demands/needs.

AADK considers working with a ‘cluster’ approach e.g. in a specific capacity area or

with similar types of partners in different countries, in order to maximise the potential

for horizontal linkages knowledge sharing etc.

AADK continue to refine its methodologies for P4C to ensure that, for example,

assessment procedures avoid the programme being used for gap-filling or for

supporting AAI core functions, unless these are resourced independently from

Danida funding. It should continue to develop new costing models to enable P4C to

facilitate but not fund placements which have implementing roles within AA offices.

5. Managing Change in AADK

In the previous section the evaluation reviewed the relevance of some of the key changes

and choices made since 2006. In this section we review how well AADK leadership has

managed the introduction and implementation of these change processes and the

effectiveness of the new governance arrangements introduced in 2009.

5.1. Revised governance AADK sees itself as a “popular, democratic organisation with its base in the Danish society”

with a membership of around 7000 individual and 51 institutional members. The current

governance structure has evolved over the last decade. In response to a Representative

Council that was seen as increasingly unrepresentative, a new system of governance was

introduced in 2005 based on a General Assembly. The new democratic model failed to meet

expectations. In 2009 a revised model of the Representative Council was introduced with the

intention of attracting a more balanced attendance, encouraging more engaged and relevant

discussion, and giving elections greater legitimacy through a web-based voting system. The

new Council - composed of a maximum of 100 individual members, representatives of

institutional members and members elected from among staff - meets bi-annually and is

responsible for the election of all Board members. The number of board members has been

Page 39: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

31

reduced from 24 to 13 (11 of whom are elected at the Council meeting; one from employees;

and one appointed by ActionAid International). The number of sub-committees has been

reduced to one - the Audit & Finance Committee. AADK ‘s identity as a membership organisation has implications for its system of

governance. Board members are elected from the membership on political rather than

competency-based criteria. Technical competence is incorporated into the system of

governance through the use of relevant sub-committees such as the Governance Advisor

and Finance and Audit Committees.

The introduction of a new model of governance has been a developmental process at a time

of significant internal changes in AADK, including its affiliation with ActionAid International,

the merger of MS and AA country programmes; and the relocation of its offices in

Copenhagen. Nonetheless, the evidence from the interviews and from a staff survey

suggests that the new governance arrangements seem to be working well; that they provide

more effective leadership and oversight to AADK; and should continue.

The general perception is that AADK now has a more diverse and engaged Council. Council

meetings are held regularly. The formats are seen to balance business, discussion and

feedback adequately. There are some minority views on the way Council deliberations are

managed; how agendas are set; and to what extent the Council should be an open forum for

wider debate. However, overall the Council is seen as a more accessible, informal and

representational process than operated in the past. Council members are still adapting to

their responsibilities and there was general agreement that Council members be given

support to strengthen their governance role.

The current Board is seen as effective. Board membership is balanced with respect to

gender and age (although it is relatively youthful compared to Boards of similar-sized INGOs

elsewhere). Board roles and responsibilities are clear. Board members seem to understand

the distinction between governance and management and do not over-interfere in

operational and management matters. Board attendance is good and members actively

engage in other AADK events. In general there is a positive view of Board processes such

as timely distribution of papers, chairing, meeting discipline, scheduling and timing. Most

interviewees felt the way that the Board monitor the AADK’s performance, appraised the

Secretary General was appropriate and effective.

Most respondents commented that Board discussions were marked by constructive dialogue

and friendly relations. It was seen as fairly open. Good Board relations were reinforced by

regular Board retreats or overseas visits. Board members appeared committed and

engaged. There was some concern that Board members need to have more induction from

existing members to help them better understand their role and the issues that AADK faces.

The Chair is seen internally to play an effective and appropriate role in managing meetings

and liaising with AADK’s management and the wider public. However, some external

respondents felt her role as AADK Chair was not compatible with her active, high-profile,

party-affiliated role outside the organisation.

There were mixed views on some specific aspects of Board working such as the balance of

Board deliberations; the quality of Board papers; and the Board review processes. In

particular, there was a view that meetings were too agenda-driven and operational and that

more time should be available for strategic discussions. There was also some concern about

the quality and scope of Board relations with AAI.

Page 40: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

32

In summary, the new governance arrangements are viewed positively so far, but are still in a

developmental phase. AADK should consider how they might evolve and be strengthened in

order to meet the challenges of the new strategy period. The evaluation recommends that:

Board members are supported to gain exposure to AAI through field visits, AAI

events, governance focal point network and other opportunities.

More time is allocated for the Board to discuss strategic issues. In particular that they

are adequately involved in the development of the new AADK strategy.

Consideration be given to an appropriate mix of skills and experience e.g. in senior

management, media and communications, in the election of new Board members

and creation of governance sub-committees.

5.2. Change management

AADK has embarked upon a wide range of transformative changes in recent years including

refocusing its programme work; redesigning its development worker programme as People

for Change; revamping and expanding its training work through Training for Change and a

network of Global Platforms; relocating its offices in Copenhagen; and perhaps most

fundamentally deciding to affiliate with ActionAid International. Each one of these initiatives

presents a management challenge but implementing these complex changes together has

involved a major, sustained change process. The evaluation drew upon 20 interviews and a

survey of a representative sample of 29 current and former staff to gather views on how

effectively AADK senior management has led change management processes in recent

years.

The general perception is that the management team handled these complex changes well,

offering a clear rationale and vision for the changes. The change process was characterised

by proactive leadership that demonstrated a willingness to take decisions rather than delay

the process. Findings from the staff survey tended to be more nuanced than the generally

positive perceptions gathered from interviewees. For example, some staff commented on the

way the change processes were communicated and the degree to which all staff were

treated fairly. However, the survey did not indicate any major criticism or disaffection with the

way the change process was handled. AADK staff generally think that the right strategic

choices have been made in recent years. In particular, most staff view MS affiliation with AAI

as a positive step, offering access to a global network and to the potential for greater impact

and influence. A key lesson to be learnt from the process has been the importance of a

shared commitment of the Council, Board Members and the Leadership Team to the

changes.

The process of affiliation and the merger of MS and ActionAid country offices were seen

generally as having been managed successfully although it was not without some concerns -

for example, regarding the compatibility of both organisations systems and approaches. The

process of integration at country level was conducted at different speeds according to local

context, capacity and readiness. The appointment of an experienced and credible change

process manager was also important in managing the merger process quickly and

comparatively positively.

While the processes of affiliation and merger were seen as a positive step by most of those

interviewed and surveyed there was some evidence from interviews that more needs to be

done to facilitate the ‘psychic integration’ of AADK staff with AAI, in addition to systems and

processes.

Page 41: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

33

The Leadership Team is seen as capable and communicating adequately, making decisions

in a timely and responsive manner, and communicating effectively across the organisation.

AADK has a relatively flat structure with limited delegation of authority or responsibility.

Some staff advocate greater delegation and spreading the managerial load, while others are

concerned that this would create a layer of “middle managers” which would jeopardise the

close relationship with senior management.

AADK’s finance and management information system is adequate but the evaluation has

encountered difficulty in extracting useful management information. This casts a doubt

whether reliable, timely data is available to managers in suitable formats in a range of areas.

More needs to be done to integrate and align the system with AAI’s systems and processes.

Two areas in particular emerged in the evaluation that will require senior management

attention. First, there appears to be a growing perception that the Leadership Team needs to

adopt a new management style. There was evidence of staff perceptions that it lacks

cohesiveness; that there is insufficient coordination between departments; and a tendency

for senior managers to dedicate themselves to bilateral, operational relationships. There was

a desire for a more cohesive approach to leadership and for the management team to focus

collectively on the strategic issues facing the organisation rather than operational and

tactical issues.

Secondly, there remains an air of uncertainty among staff. While past changes are positively

perceived, AADK remains an organisation in transition. Interviews revealed some concern

about the future direction AADK might take and survey results, for example, revealed a

comparatively low level of confidence among ‘academic’ and office staff in the future

direction of AADK. There are high expectations that the new strategy will provide such

direction and, for example, clarify AADK’s identity and profile in Denmark.

In conclusion, AADK has demonstrated leadership, vision and drive to manage

comparatively successfully a process of radical change in recent years. To sustain this

momentum and embed these changes successfully over the next strategy period, AADK

should:

Invest in building a more cohesive leadership team with more of a strategic than

operational focus.

Address leadership ‘depth’ and development in AADK e.g. delegation, succession

planning. The Leadership Team may need to delegate more operational aspects of

its work and allow coordinators to play a more managerial role.

Improve finance and management information systems to inform decision-making.

Ensure the AADK strategy development process is inclusive and provides a clear,

confident of the organisations identity and future.

Consider how best to document and disseminate the learning from AADK’s change

processes including affiliation with AAI - for example, as a case study of change

management case study and/or review of wider changes in Northern and Southern

civil society.

6. Conclusions

The last six years have been a period not only of transition, but of transformation in the

history of MS/AADK. Widespread, profound changes have been introduced in nearly all

Page 42: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

34

aspects of the organisation. As with all such significant change processes, the

transformation in AADK in recent years has not been without its casualties and critics.

Nonetheless, it is remarkable that current and former staff interviewed and surveyed

predominantly perceive that the right decisions were made; that senior management

implemented the change process openly and fairly; and that the organisation is heading in

the right direction.

The evaluation agrees that, in broad terms, the strategic choices made by AADK were

relevant and appropriate in light of evolving trends in global and Danish civil society and the

donor community. For example:

The decision to adopt a ‘democracy focus’ in its programming has been vindicated by

the subsequent evaluations of country programmes which have confirmed their

relevance and effectiveness. It has also enabled AADK to build on its competence in

this area to demonstrate a specific added value in programming to both AAI and

Danida. AADK leadership of the Just and Democratic Governance theme has

confirmed within the federation AADK’s key competence in this area and contributed

to the objective on ‘holding governments to account’ being included as an objective in

the new AAI strategy.

The launch of People4Change has enabled AADK to evolve from the outdated

concept of a development worker model to experiment with a demand-driven, flexible

model of development cooperation that resonates with a growing interest in the North

and South with people-to-people collaboration.

The Training4Change programme has demonstrated its popularity and effectiveness

by delivering an expanded range of training products to a growing client base who

rate the quality of training highly.

Volunteers, host organisations and families express high levels of satisfaction with

Global Contact which continues to translate AADK’s traditional values of inter-cultural

understanding and solidarity in a contemporary setting.

The decision to assume leadership of the Activista programme in AAI revived the

project and has reinforced its youth activist profile and campaigning capability in

Denmark.

The revised system of internal governance in AADK is working well and has

contributed constructively to senior management’s leadership of change and strategy

development.

The most significant strategic decision was, of course, to become an affiliate of ActionAid

International. Becoming a member of a global federation offered AADK the opportunity to

extend its global reach, influence and impact. AADK succeeded in negotiating a leadership

role in two programming areas and agreeing to pilot its ‘heritage’ programmes P4C and T4C

within the federation to assess how they might be mainstreamed in the future. This

evaluation concludes that there is good evidence to support the conclusion that AADK could

continue to play a valuable leadership role within AAI in these two areas and that P4C and

T4C have a positive contribution to make to the new AAI strategy. AADK will need to enter

into dialogue with AAI to explore whether Global Contact can contribute to the new strategy

as a ‘business’ or social programme.

The previous sections have highlighted the impressive AADK achievements during this

period of significant change. The evaluation has also identified a number of challenges that

will need to be addressed as AADK develops its new strategy. One way of thinking about

these challenges is about how AADK will manage its multiple ‘identities’ or relationships with

Danida, Danish civil society including its own its membership, AAI and Southern civil society.

Page 43: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

35

Relationship with Danida

AADK’s relationship with Danida requires special consideration given its level of financial

dependence on Danida funding. Recent trends in the sector associated with the Paris

Agenda have had implications for Northern NGOs and suggest some potential challenges to

AADK in the next few years. It may be that Denmark does not follow these trends as

enthusiastically as some Northern European governments (since, for example, the Minister

for Development Cooperation in the new left-of-centre Danish government is from the

development NGO sector). Nonetheless, the evaluation thinks it more likely that a

combination of trends - the opening up of frame funding to new entrants; the linking of

funding to performance and fit with donor strategy/priorities; and the tendency to more direct

funding through embassies or multi-lateral trust funds in the South - may affect AADK’s

Frame agreements with Danida during the next strategy period.

This would suggest that AADK should consider additional funding models to sustain the new

strategy - for example, through other institutional donor programme funding, alternative

income-generating initiatives and increased public fundraising in Denmark (though there

may be some limitations on this income stream). It is also a plausible scenario that AADK

will be required to report more effectively on its added value in strengthening Southern civil

society. This has a number of implications among which will be the need to demonstrate its

benefits to Southern civil society beyond AAI. This will require revisions to AADK’s current

management information and reporting systems.

Relationship with Danish civil society

AADK has successfully developed an agency profile in Denmark associated with youth

activism. This public ‘identity’ is reinforced by its campaigning, and its work with Activista and

Global Contact. It is also reflected in the membership of its Board. Its new premises in

Copenhagen and the close proximity of the Danish Global Platform communicate a youth

and activist ethos. This has a strong resonance with the new ActionAid strategy and with the

‘spirit of the times’ associated with the ‘Arab Spring’. By association, there are some

indications that AADK is gaining a reputation within the Danish NGO sector for innovation

and creativity.

At the same time, AADK is a middle-aged rather than a young organisation with a strong

heritage and culture. Some of its continuing core programmes such as P4C and T4C draw

upon that heritage. Its core competencies in governance to date, for which it is beginning to

be recognised in Danida and AAI, are those of research and strengthening capacity, not of

activism. These are the areas that absorb most Danida funding. AADK also has a profile in

Denmark as a well-established, ‘reputable’ development NGO and draws upon a

constituency of support among the middle aged. While AADK is dialoguing with the ’young’ it

is fundraising from the ‘old’.

There is clearly, as revealed in the staff survey, some uncertainty within AADK about how

this dual profile of ‘youthfulness’ and ‘seriousness’ will be managed in the future and which

of these aspects will predominate. Both are assets to AADK, but may be in tension. AADK

will need to manage its programme strategies and public communications sensitively and

intelligently in the next strategy period to ensure that different approaches (such as activism

and capacity building) do not work at cross purposes.

Relationship with AAI

The new AAI strategy offers an opportunity to AADK to add value to the federation -

particularly through the incorporation of governance and youth mobilisation as strategic

Page 44: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

36

objectives and the recognition that Member affiliates can lead international work. The

evaluation provides evidence of a largely positive track record of achievement within the

federation to date. Early discussions on the new AAI strategy indicate that AADK will be able

to continue to fulfil a role in promoting and supporting governance work in the federation,

and that there is an openness to on-going collaboration with T4C and P4C.

The evaluation demonstrates that AADK has the competencies to extend the geographical

range and impact of its programmes through its affiliation with AAI. However, management

arrangements outlined in the MoUs between AAI and AADK e.g. on governance, Activista.

P4C and T4C, have been less than fully effective. It will be important that delegated

responsibilities to a Member such as AADK are based on a clear mandate and system of

accountability to the federation that allows Members to lead international work and

communicate their activities across AAI with confidence.

Relationship with Southern civil society

The decision to affiliate with AAI offers AADK opportunities to increase its impact and

influence. It also has implications for its relationship to Southern civil society. The process of

merging MS and AA country offices has had an impact on AADK’s relationship with Southern

CSOs and its concept of partnership. AADK now has partnership arrangements with AA

country offices rather than directly with independent CSOs. Clearly, poor women and men

remain the intended beneficiaries of AADK programming and Southern CSOs are key

partners in this mission. AADK also continues to have contact with Southern civil society.

Funding is channelled and MOUs are signed, however, with AA country offices. There is

evidence also that programmes such as P4C and T4C are increasingly orienting themselves

as service providers to AA offices. This need not mean that the majority of ‘beneficiaries’ of

these programmes - for example, participants on TCDC training or host organisations for

Inspirators - are not from Southern civil society. However, it does highlight the need for

AADK to demonstrate - and indeed quantify - the links with and benefits to independent,

diverse Southern civil society. This is needed to support its public fundraising; to feed and

legitimate its campaigning; and to meet the needs of its major donor, Danida.

In conclusion, AADK has made some impressive achievements during a period of wide-

ranging and profound change. But it remains in a period of transition. The new AAI strategy

offers a number of opportunities for AADK to increase its impact and influence. The new

AADK strategy will need to meet a number of challenges if the organisation is going to build

on these achievements. The evaluation believes that AADK’s strengths are focused around

two approaches to delivering its core competence in governance:

Connecting people, particularly young people, to build a global civil society to fight

poverty e.g. Global Contact, Activista.

Strengthening capacity of Southern civil society e.g. T4C, P4C to help make

governments more accountable to poor and marginalised people. .

Both these ways of working are relevant to AAI strategy “Peoples Action to End Poverty” and

to the strategy objective of ‘Holding Governments to Account’. AADK will be at is most

effective when it retains a clear focus and achieves maximum synergy between the two

approaches in its programming.

Page 45: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

37

Annex 1: Objectives and scope of the evaluation

The overall objective of the evaluation is to:

Give an overall assessment of the relevance of the decisions and direction taken by

MS/AADK over the past 6 years vis-à-vis the general developments in the external

environment - including Danish CSO strategy.

The specific objectives of the evaluation are to:

Assess to what extent AADK has succeeded in implementing its various strategies and

meeting annual plans and targets.

Provide input to the internal discussion within AADK and to AADK’s dialogue with AAI and

with Danida on the strategic and programmatic direction of AADK (the new strategy) over the

coming six years

Provide evaluation inputs as agreed in the MoUs with AAI within the fields of T4C, P4C, the

Governance theme and Activista.

The evaluation will comprise, but not necessarily be limited to the following:

1. Assessment of AADK’s role and strategic placement in the international and the Danish context in

relation to the Danish Civil Society Strategy and the general global trends in CSO development.

2. Building on the various reviews of AADK supported activities in the programme countries and

notably the recent evaluation of all Country Programme Strategies (June 2011); the review of the

MENA programme, and the review of ‘Recycling for Development’, the team shall analyse the

findings and recommendations from these reports and discuss them with key stakeholders. On

this basis, the team will synthesize and assess the relevance, effectiveness and lessons learned

from the programme areas covered by these reports. With regard to Recycling for Development

the team will assess the strategic coherence between this and other AADK activities and advise

on possible future modalities for continuing this type of activity.

3. Assessment of 1) AADK’s role in leading the international governance theme; 2) the

implementation of plans and the relevance and quality of work (including concept development,

manuals and training); 3) the performance of the International Governance Team as assessed by

AA country office staff and key informants; 4) to what extent have the objectives of the

management contract been met, 5) the lessons learned for the potential of delegating AA

international work to members. The team will make recommendations towards a potential

continuation of AADK playing a leading/supporting role within the field of just and democratic

governance in AAI.

4. Assessment of the Training4Change programme looking at 1) the relevance of the different

courses (at TCDC and Global Platforms) in relation to the AADK strategy; 2) the reaction,

learning, behaviour, and results of the training as judged by participants (tracer study); 3) the

business model – including cost and cost recovery, market and marketing and growth potentials,

professional capacity of trainers. The team will make recommendations and present options for

the development of T4C and possible future interface/integration with AAI.

5. Assessment of the piloted People4Change programme – Advisors and ‘Inspirators’ – to establish

relevance, effectiveness and efficiency in relation to capacity building of partner organisations in

the South. The study will include a comparative study of other AAI experiences with people to

people activities, notably the ‘Fellowship Programme’ in India and Myanmar. The team will make

suggestions and present options for continuation and possible mainstreaming of P4C activities in

AAI.

Page 46: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

38

6. Assessment of Global Contact – 1) the business model; 2) satisfaction and learning by

volunteers (tracer study – also assessing impact on their lives); 3) level of appreciation and sense

of usefulness by host partner organisations and families (questionnaire and tracer study). The

team will make recommendations on the further development of the programme – and possible

mainstreaming in AAI.

7. Assessment of AADK’s campaign and press/media work over the past three years; 1)

relevance in relation to strategies; 2) planning, monitoring and evaluation; 3) involvement of

volunteers; 4) target groups; 5) effects over the past three years.

8. Assessment of the development of Activista over the past two years – 1) AADK’s role as a

coordinator; 2) effectiveness in relation to the management contract and annual plans; 3) lessons

learned in relation to delegating AA international work to members. The team will make

recommendations towards the potential continuation of AADK playing a leading/supporting role

within Activista.

9. Assessment of the composition of AADK’ fundraising programme – 1) how well have plans

been developed, tested, improved and implemented; 2) how can the synergies between the

different activities be optimized; 3) the quality of the communication with members, supporters

and donors.

10. Assessment of the overall management of AADK – both in terms of change management in

AADK in general and in relation to the various activity areas mentioned above.

11. Assessment of the functioning of the governance structures of AADK – 1) the establishment

and functioning of the Council, 2) the role and functioning of the Board, 3) the possibilities for

active members to influence the organisation.

Source: pp42, 43 of Tender for Evaluation of ActionAid Denmark

Page 47: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

39

Annex 2: Evaluation surveys: methodologies Twelve surveys, each addressing specific groups of respondents, were carried out to inform the

evaluation as a complement to other data collection methodologies. The following table provides an

overview of the surveys and which elements of the evaluation they informed.

Table 1: Summary of surveys

Survey Title Evaluation Area and Purpose Sample Framework

Tracer Study

1 (TS1)

Global Contact & Training for Change

The tracer study targeted youth

volunteers who had taken part in Global

Contact. It provided data on participant

experiences of GC, and information on

how GC had affected their life choices,

especially social activism

A sample of volunteers from the 2006 and

2009 cohorts were asked to participate in

the survey. The 2006 cohort gave us data

on changes over a longer period of time.

The 2009 cohort were the first to receive

obligatory training at the Global Platforms,

so provided data on the impact of the

training. This was the only survey where

we took a random sample: 140 from the

2009 cohort and 50 from the 2006 cohort

Tracer Study

2 (TS2)

Training for Change (Global Change).

The tracer study targeted youths who

had taken the training. It provided data

on the impact of training on their life

choices, and particularly whether they

had become Activistas

Included in the survey were all participants

from Global Platforms in Denmark, Nepal

and Tanzania who had received training in

2009 and 2010. We did not include those

receiving training in 2011 as it would be

too recent to allow any degree of tracing.

Survey 1

(S1)

Global Contact & Training for Change

The survey targeted organisations which

had hosted Global Contact volunteers

with a view to ascertaining the

usefulness and impact of volunteers on

host organisations, and the relevance of

the preparation and training of

volunteers.

The sample frame was organisations

which had hosted volunteers in 2010.

Originally we anticipated a high number of

organisations from across a range of

countries. We planned to do a stratified

random sample, with stratification by

geographic location and type of host

organisation. However, it proved extremely

difficult to obtain the necessary data on

organisations in the timeframe available,

which severely limited the sample frame.

Survey 2

(S2)

Training for Change (Training of

Trainers)

The survey targeted participants of ToT

courses trained at Global Platforms in

Denmark, Jordan and Tanzania, with a

view to ascertaining the relevance and

usefulness of the course, including

whether participants went on to become

trainers on behalf of Action Aid.

The sample framework was all participants

from 2009 and 2011 (no courses were run

in 2010).

Survey 3

(S3)

Training for Change (Governance

courses)

This survey targeted organisations

which had commissioned governance

courses or sent staff members on

governance courses. The aim was to

The sample frame was organisations that

had commissioned courses or sent staff on

courses in 2010. The original aim was to

do a random stratified sample, with

stratification by geographic location and

specific courses. The final sampling frame

Page 48: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

40

assess course relevance and

usefulness, and the impact on

organisations – notably on governance

work.

was determined by availability of useable

data on organisations.

Survey 4

(S4)

Training for Change (Governance

courses)

This survey targeted participants on

governance courses. The aim was to

assess course relevance and

usefulness, and the impact on

organisations – notably related to

governance work.

The sample frame was individuals who

had taken governance courses in 2009

and 2010. We anticipated a sample frame

of around 3500 (including other courses),

and planned a stratified random sample of

500, with stratification by geographic

location and specific courses.

Survey 6

(S6)

People for Change: Advisors &

Training for Change

This survey targeted all individuals who

had taken up placements as advisors

since 2009. The aim was to assess the

usefulness and impact of advisors, as

well as the systems in place for

appointment and preparation, etc.

There were only a small number of

individuals, so the sample framework was

all advisors

Survey 7

(S7)

People for Change: Inspirators &

Training for Change

This survey targeted all individuals who

had taken up placements as inspirators

since 2009

There were only a small number of

individuals, so the sample framework was

all advisors

Survey 8

(S8)

People for Change: partner

organisations

This survey targeted AA partners who

had received inspirators or advisors, to

assess the impact.

The survey was sent out to partners by AA

regional offices rather than INTRAC. The

survey was completed in Word, with

options of completing it in English, Spanish

and Portuguese.

Survey 9

(S9)

Country Directors

This was a longer survey designed to

gather views from Country Directors on

a wide range of issues: International

Governance, training, and programmes

such as People for Change.

The survey was administered to all

Country Directors.

Survey 10

(S10)

Governance Focal Points

This survey was targeted at Governance

FPs. The aim was primarily to gather

data on the effectiveness of the

International Governance Team.

The sample framework was all

Governance Focal Points.

Survey 11

(S11)

AA/MS Denmark Staff

This survey was sent to a sample of

MS/AA DK staff with the aim of

assessing the implementation of

changes to the organisation in recent

years.

The sample framework was a select

sample of current and former staff,

including senior management, former

country directors, and academic staff.

Page 49: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

41

Survey design

All surveys, with the exception of S8 (P4C partners) were designed as on-line surveys. The majority

used the online Survey Monkey software (S7 used Web Survey Creator as this was multilingual and

Survey Monkey cannot host multilingual surveys). All bar S8 were administered by INTRAC Oxford.

The surveys were designed by a team based in Oxford (Rachel Hayman with three research

assistants), in close consultation with Evaluation Leads. Draft surveys were commented on by

relevant staff members at MS/AA Denmark, as well as the evaluation Reference Group.

Small pilots were conducted for most of the surveys. These were completed by individuals suggested

to the survey team by MS/AA Denmark. Surveys were revised as necessary following the piloting.

Sampling

Data for the sampling framework for all surveys was provided by staff at MS/AA Denmark (HQ, TCDC

and country offices). We were initially given approximate numbers for total populations from which to

establish our sampling framework (i.e. the recipients who would receive the surveys). It was extremely

difficult to get total population sizes for many of the surveys as that information was not readily

available within the timeframe, so this information is very rough in Table 2 for some surveys.

MS/AA Denmark provided us with contact details of prospective survey respondents. This process

was somewhat arduous, as we had to chase MS/AA staff to get. An extreme example was S1, which

targeted organisations which had hosted Global Contact volunteers. MS DK did not have this

information readily to hand and required time to compile it (affected also be holiday). We initially

anticipated (following the inception meetings) that the total population could amount to around 200

organisations, we received useable information on only 31 organisations. We also had to proceed to

launching the survey without first conducting a pilot as the data were received very late.

This sort of process meant that our sample selections were determined by the data that MS/AA was

able to provide within the timeframe. The total number of potential respondents was often much

smaller than the initial estimates, as AA/MS Denmark did not often have the data easily to hand. In an

ideal world - referring again to the example of S1 – we would have received a full list of all host

organisations from which we could have selected our sample using a random sampling or stratified

random sampling technique. As it was, we were bound by the number of organisations about which

MS/AA Denmark had information. We must conclude, therefore, that the data for a number of the

surveys are biased and the samples are convenience samples (and therefore less scientifically

robust) rather than probability samples. Surveys 6, 7, 9, 10 are more robust in this respect as the

sample frameworks covered all possible respondents.

Moreover, the data was often problematic and required ‘cleaning’ to remove duplicate names and

invalid email addresses, resulting in much smaller final samples. Table 2 summarises the sampling for

the surveys.

For all surveys except TS1 and S4, the survey was sent to ALL possible respondents following data

cleaning. For TS1 we did random sample for each cohort (2009 and 2006 separately) using a random

sample generator (randomiser.org). The S4 we stratified the data, to only include participants on

governance courses, as the lists provided also included participants on other courses.

Table 2: Sampling

Survey Title Total

Population*

(all years)

Sample framework (data

provided by MS DK for sample

year groups)

Sample Size (number of recipients following

sampling, data clean-up, and piloting – pilot

respondents excluded)

TS1 000s 535 (2006 and 2009 cohorts) 190 (140 – 2009 cohort; 50 – 2006 cohort)

TS2 ca. 280 97 (2009 & 2010) 95

S1 00s 50 (2010 host organisations,

apparently about half the total for

31

Page 50: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

42

that year)

S2 ca. 150 76 (2009 & 2011) 74

S3 data not

provided

145 (2010) 142

S4 ca. 3500 974 (2009 & 2010) 485

S6 29 29 29

S7 72 72 67

S8 ca. 100 95 95

S9 35 34 33

S10 56 56 55

S11 29 29 29

* Total Population is the initial estimated numbers for TS1, TS2, and Surveys 1-4, and S8.

Response rates

Surveys were sent out on the 26th and 27

th of November, with an initial deadline for responses of 4

th of

November (ca. 10 day window). Reminders were sent out. After an initial analysis of response rates, it

was decided to push for further responses. MS/AA Denmark staff sent out targeted emails to non-

respondents in an attempt to boost the response rates. Two surveys were closed on 7th November

(TS2 and S11), and the remainder on 10th November.

Table 3: Summary of the response rates to each survey.

Survey title Sample size Responses (absolute) Response rate (%)

TS1 190 61 31.5

TS2 95 33 34.7

S1 31 16 51.6

S2 74 29 39.2

S3 142 22 15.5

S4 485 111 22.8

S6 29 18 62

S7 67 38 56.7

S8 95 65 68.4

S9 33 10 30

S10 55 15 27.2

S11 29 28 97

There are two main explanations for low response rates:

Individuals choosing not to respond, which includes those who formally ‘opted out’ out of the

survey or who emailed us to say the survey was not relevant to them.

Surveys that did not reach recipients due to invalid email addresses. This particularly

affected:

o S2, where there was a delivery failure for 36 out of 74 respondents (48%)

o S3, where there was a delivery failure for 27 out of 142 respondents (19%)

While response rates overall were low (with only three surveys reaching over a 50% response rates),

we can consider the responses that were received as representative, although there are potential

biases to be taken into consideration (see below).

Page 51: ActionAid Denmark Evaluation - Mellemfolkeligt … · ActionAid Denmark Evaluation Synthesis Report Cowan Coventry 13th January 2012 INTRAC Oxbridge Court, Osney Mead, Oxford, OX2

Evaluation of ActionAid Denmark | Final Synthesis Report | INTRAC January 2012

43

Analysing and interpreting the data

Data analysis methods have been fairly simple, using functions available within Survey Monkey and

Excel to extract and disaggregate data.

Issues to be aware of:

Non-responses in certain surveys to particular questions do affect the results (e.g. in TS1 we

had a total response of 61, but 5 respondents only completed the first question, and there

were consistently around 16 non-responses to most questions. S10 had 15 responses, and

an average of 3 respondents skipped each question.

For drawing inferences from the data, it is important to bear in mind the sample size and

absolute response rates for particular questions. So a percentage response of 33% may

represent the viewpoint of 1 respondent out of 3 who answered that question. This is pertinent

for the surveys with smaller populations.

Sampling biases, due to convenience sampling for many of the surveys, and low response

rates.

Software: Web Survey Creator used for S7 failed to record qualitative responses to some

questions. S8 was administered in Word version, then had to be inputted manually into

Survey Monkey for analysis. There were a number of complications arising in this process

which had to be addressed (e.g. ineligible or incorrect responses).

Triangulating the data from the surveys with data gathered using other methodologies will enhance

our confidence in the validity and reliability of the survey data, and therefore the associations that we

can plausibly infer from the data.

Key limitations

Key limitations arising in the survey process are:

Data biases (as explained above)

Timeframe: while the timeframe for administration of the surveys was sufficient, the time was

short for designing and piloting the surveys which left little room for manoeuvre for reflection

and redesign of survey questions. S9, for example, was halted and re-administered after

problems were discovered in the structure – more robust piloting would have averted this.

Likewise the timeframe was too short to deal with anomalies in the data on potential

respondents received from MS/AA Denmark which meant – where there were problems with

invalid emails we could not attempt to check addresses but just had to exclude these

participants.

The timeframe also meant that survey data was being gathered and analysed at the same

time as other data was being gathered. Questions arising from the survey data could not be

followed up through other data collection methods, e.g. one-to-one interviews.

The time-span for the tracer studies was only a few years between the Then and Now.


Top Related