evaluating effectiveness to what effect? joanne sharpe, office of development effectiveness, ausaid

Evaluating EffectivenessTo what effect?Joanne Sharpe, Office of Development Effectiveness, AusAID

What does ODE influence?• Systems, processes and policies within AusAID and

other government agencies, which guide the effectiveness of the overall program.

• Staff knowledge and understanding of effective practice, so that mid-level program managers and desk officers are aware of and can apply lessons from ODE’s evaluations to their own work.

• Broader knowledge and debate about development effectiveness for a wider community of development practitioners.

Review approach• Purpose:

– Define a working set of ODE ‘influencing objectives’

– Map political and operational context– Identify influential stakeholders– Identify strengths and weaknesses of engagement

• Methods– Internal ODE workshops– Semi-structured interviews (AusAID, other Gov.

agencies, other donors, research institutions)– AusAID staff survey (n=84)

Building alliances“How much visible senior management support does

ODE have? Are you ensuring ODE is relevant to the current priorities of senior management?”

• Early and consistent engagement with end users is critical if work is to have influence.

• Violence against women (2008): contributing evidence as part of a broader movement for change

• Law and justice evaluation (forthcoming): Building new coalitions through intensive engagement

Engaging busy staff (1)• ODE has influence over staff through the course of

evaluations: 72% of staff involved in evaluations said it was a good learning experience.

• Mixed picture about the ‘relevance’ and ‘usefulness’ of ODE reports and other products.– 94% staff know ODE products and rate them

‘moderately useful’ to ‘very useful’– About half of the key informants interviewed said

products were not relevant enough to make time for– Sector and performance-focused areas within AusAID

were more likely to use ODE products than country programs

Engaging busy staff (2)• Finding ways to improve accessibility and

‘digestibility’ a key challenge for ODE– 60% of respondents wanted to see more ‘short

summaries of long reports’; 70% wanted more good practice examples

– Appetite for regular communication about new products and current work.

– Internal AusAID communications channels used.

Contributing to broader debate• Beyond AusAID and the Australian Government,

there are communities of interest and practice who will seek out evaluations and information about aid effectiveness.

• Engagement with external organisations create networks for dissemination

• Knowledge repositories and the blogosphere are important areas for growth

• External promotion creates a multiplier effect for internal audiences

Where to from here?• Post Independent Review of Aid Effectiveness – more work

for ODE?• Improving accessibility and digestibility:

– ODE Briefs– ODE Talks podcast series– Website overhaul

• ODE Performance framework– How are ODE’s products used, to inform policy, practice

and debate?– Is ODE meeting standards for product quality,

engagement, accessibility, timeliness and transparency?

ContactJoanne SharpeInfluencing and Communications ManagerOffice of Development Effectiveness

joanne.sharpe@ausaid.gov.auwww.ode.ausaid.gov.au

Visit our stand in the Exhibition HallSearch for ‘ODE Talks’ on iTunes!