aflatoun quality assurance and impact assessment strategy (aqis) strategy overview amsterdam, august...

27
Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Upload: frederick-barrett

Post on 27-Dec-2015

221 views

Category:

Documents


5 download

TRANSCRIPT

Page 1: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Aflatoun Quality assurance and Impact assessment Strategy (AQIS)

Strategy overview

Amsterdam, August 2008

Page 2: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Contents of this document

This working document outlines Aflatoun’s Quality Assurance and Impact Assessment strategy. It is a working document and as such under revision of the AQIS committee and the Taskforces for Quality Assurance and Impact Assessment.

The document is divided in five parts; sections 2 – 4 provide the key elements of AQIS:

1.Strategy overview – Principles and structure

2. Quality Assurance

3. Output Tracking & Outcome Assessment

4. Impact Assessment

5. Planning

Page 3: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

1.1 Strategy overview – Principles and structure

As Aflatoun scales-up rapidly to reach its 2010 goals of bringing its Child Social & Financial Education programme to over 75 countries by 2010, it is in need of innovative and strategic approaches towards assuring the quality and impact of its programmes. Our strategy involves a variety of evaluation tools and stakeholders of our programme. AQIS follows three principles:

 I. Strategic alignment with Aflatoun’s Theory of ChangeII. Progressive learning through integrated knowledge management of the involved evaluation

toolsIII. Effective involvement of evaluation experts and practitioners 

The strategy for quality assurance and impact assessment has been designed on the basis of extensive discussions with our implementing partners and international stakeholders at our Regional and International meetings. The current document has included the comments and recommendations done at the first Impact assessment Taskforce meeting in August 2008. The strategy is outlined below along the lines of the three principles given above.

Page 4: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

4

I. Strategic alignment with Aflatoun’s Theory of Change

Aflatoun’s strategic measurement of quality and impact is aligned with our programme’s Theory of Change (see next page), defined by the Aflatoun Secretariat in conjunction with our Social Return on Investment (SROI) experts, our implementing partners and the impact advisory committee.

The elements of AQIS are designed such that they monitor and evaluate the different effects of the Aflatoun educational programme and advocacy. Though the elements are presented as separate categories, they work together naturally and powerfully. Quality means nothing without Impact, and Impact can be dangerous without Quality. The learning of each element is constantly fed back into the activities and research of the other elements, and vice versa. A. Quality Assurance: Change through education starts with high-quality organization and processes. In order to assure the maximum quality of the entire Aflatoun Programme and Advocacy, a range of quality tests at partner and Secretariat level are implemented. B. Output tracking: Aflatoun desires to understand the scope, outreach and activities comprising the Aflatoun Programme. The Secretariat facilitates all partner organizations with tools for output tracking.  C. Outcome Assessment (Evaluation): Aflatoun desires to understand the scope, outreach and activities comprising the Aflatoun Programme. The Secretariat facilitates all partner organizations with tools for output tracking. Through quality education, on a considerable scale, Aflatoun sets out to change children’s competencies and related behaviour. Different research tools are being developed to assess these outcomes.

D. Impact Assessment: Our mission is our ultimate goal. In order to understand whether we establish this impact, Aflatoun asks leading research institutions using cutting-edge impact assessment methodologies, to look at the impact of (elements of) our programme.

Please refer to page 6 for a schematic overview of AQIS.

1.2 Strategy overview – Principles and structure

Page 5: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

S

• I know I’m unique!• I like to explore• I am a compassionate member of the world

• I use resources responsibly• I save regularly• I spend responsibly

• I undertake enterprises• I enterprise for change• I value teamwork

• I am a changemaker• I stay in school longer• I have enough resources• I apply my capabilities• I decide when I want to marry• I am happy!

Core Element 4: Planning & Budgeting

• I know/claim my Rights• I care for others & myself

• I plan to realize goals• I can make a budget

Curriculum Quality & output

ExpectedCompetency outcomes

Expected behavioralOutcomes

• I run a micro-enterprise• I have organized/ conducted projects for social & green causes

• I make budgets • I set financial goals for myself

• I save & deposit regularly• I have a savings account• I withdraw less

• I defend my and others’ rights •I participate in democratic processes

• I confidently participate in different activities in my community

Core Element 5: Social & Financial Enterprise

Core Element 3:Saving & Spending

Core Element 2:Rights & Responsibilities

Core Element 1: Personal Exploration

• Raising global CSFE awareness• Child Friendly Banking• Creating children’s global movement

• More inclusive, participatory education• Structural banking reform • More children’s savings accounts• Global children’s community

I am economicallyempowered!

I am sociallyempowered!

vv

vv

BehaviourImpact

Advocacy

Theory of Change

Page 6: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008
Page 7: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

7

1. High quality partner selection 2. Partner training & facilitation• training• field visits • web & network sharing• twinning & peer evaluation 3. Partner Self-evaluation checklist (QA checklist)

4. Secretariat effectiveness & efficiency

1. Partner, School, class output checklists

2. Social Return on Investment (SROI)

1. Individual student questionnaires

2. Social Return on Investment (SROI)

3. Process Review & Reflection*4. Participatory Action Research projects*5. Focus group discussions*6. Real life story collection*

1. Randomized evaluation trials

2. Longitudinal studies

SROI web tool

Partner DB

QA web tool

AQIS - Structure

Impact and Outcome Indicator development (social & financial)

* Subject levels: Partner organization, school, class, student

Page 8: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

8

II. Progressive learning through integrated knowledge management of the involved evaluation tools

The learning of the different elements and tools of AQIS are linked together and spread out over the entire network.

Indicator development: Aflatoun develops a cross-cutting indicator database that aims to capture the key indicators for all parts of AQIS: quality, output, outcomes and impact. An integrated set of indicators, related to the theory of change, will ensure that the programme will be evaluated on the points that matter. (The arrow on the last page signifies this indicator development) Network sharing: The design, methodology, implementation and results of the evaluation tools are shared over a global network of partner organizations and research institutions (see more below).

Publications: Aflatoun’s key publication on AQIS is its widely distributed annual publication Children & Change, presenting the key programme results, learning, and successes and challenges. An important part of the quality assurance results will be included in our annual report. Curriculum development: As documents of what works and what does not, the quality reports, and outcome and impact studies will be key inputs for the development of the Aflatoun Curriculum and the materials in which this is laid down. This development will happen under the shared guidance of the Aflatoun Secretariat and the Curriculum Development Taskforce.

1.3 Strategy overview – Principles and structure (continued)

Page 9: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

9

III. Effective involvement of evaluation experts and practitioners Aflatoun is an organization that wants to make a change. In order also to effectively measure the

change it effects, Aflatoun collaborates with leading research experts and actively involves the people whom are participating in, and are at the heart of the programme: the children, teachers and partner organizations. For AQIS, Aflatoun consults:

 The AQIS (impact) advisory committee: An expert committee with members from a variety of

research and action fields. The committee meets bi-annually and advises on AQIS and organization strategy, and the methodology and implementation of specific research projects and evaluation tools.

 The Quality Assurance and Impact Assessment Taskforces: At Aflatoun’s global Campaign

Launch in March 2008, several focus groups of partner organizations on particular topics were installed. The Quality Assurance taskforce and the Impact Assessment taskforce respectively, consist of representatives from all four continents in the Aflatoun network. They advise and decide on AQIS strategy, evaluation tool implementation, and they provide the on the ground perspective. In August 2008 the Impact Assessment Taskforce had its first 3-day meeting in Amsterdam.

 The children and teachers: As evaluation tools are tested, a sample of our target group is

involved. Students and teachers from several of our programmes are asked for their feedback, and the tools tested on them before they are applied to the entire programme network.

 Aflatoun global network: Via network-wide, regional and sub-regional meetings and individual

partner visits, topics from AQIS are being discussed. Further means of sharing are the monthly Aflatoun newsletter, the online member area a two-weekly Secretariat email update for partner organizations, and day-to-day communication.

1.4 Strategy overview – Principles and structure (continued)

Page 10: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Aflatoun Quality assurance and Impact assessment System

1.Strategy overview – Principles and structure2.Quality Assurance3.Output Tracking & Outcome Assessment4.Impact Assessment5.Planning

10

Page 11: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

2.1 Quality Assurance

Tool: High quality partner selectionExplanation

Key Objectives Ensuring high quality partner organizations within Aflatoun network.

Items covered - Due diligence check- Quality standards for partner admission

Who do we do this for? Donors, Aflatoun partner network.

Who executes? The Secretariat selects partner organizations for admission to the Aflatoun network and collects the relevant information. The Secretariat consults the Quality Assurance Taskforce and a combined advice will be put before Aflatoun’s General Board.

Further description - In the initial pilot phase, Aflatoun screened partners via pre-existing networks such as Ashoka (www.ashoka.org). A mixture of individual social entrepreneurs, small NGOs and global NGOs were selected to create a rich mix of learning and programme development. - Via formalized internal partner selection procedures the Secretariat currently streamlines the selection process and ensures that the network consists of high quality partner organizations. - Partner selection focuses on:

- due diligence standards- capacity and willingness of potential partner organization to align with Aflatoun’s mission, vision, and strategic objectives.

Projected timeline (also see planning)

- ongoing

11

Page 12: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

2.2 Quality AssuranceTool: Training & FacilitationExplanation

Key Objectives • Ensuring high quality programme implementation• Ensuring optimal facilitation of partner organizations through training, field visits, network sharing

Items covered - Sub regional training, teacher training- Field visits- web & network sharing- twinning & peer evaluation

Who do we do this for?

Aflatoun partner network, donors

Who executes? Aflatoun Secretariat sets up the training, field visits, network meetings and web facilitation in cooperation with the Aflatoun partner network. Twinning is undertaken by the partners in our network.

Further description 1. Training: Training of partners, trainers and teachers is crucial for the quality of Child Social and Financial Education. Aflatoun promotes a training tree model (cascade training). The Secretariat trains the implementing partner, who trains the trainers. Trainers, in turn, train teachers/professionals working with children. The Secretariat trainers have been trained by a professional training consultant.A training kit has been designed to:• Facilitate the appreciation for the experiential/interactive learning style as promoted by the Aflatoun Programme through the use of the same methodology• Enable the learning of the Aflatoun curriculum and its five core elements• Evaluate the participants level of learning at the end of the trainingAdditional evaluation activities are conducted through visits to schools and the conduct of impact assessment activities.2. Field visits: The target is that each partner programme is visited by Aflatoun staff once before 2010 for full review and reflection. In as many cases as possible, this trip will be linked to another country visit, so as to minimize additional time and expense3. Web & network sharing: Development and sharing of digital member sharing space (member area), bi-annual regional meetings, bi-annual international consultations.4. Twinning of Partners: This includes site visits between partners to learn from each other about the programme, to share experiences, lessons learned and best practices.

Projected timeline (also see planning)

1. Key training for partners happens at bi-annual regional and sub-regional meetings; further programme training provided by trainers happens throughout the year.

2. 75 countries will be visited throughout the period 2008 – 20103. ongoing 12

Page 13: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

2.3 Quality Assurance

Tool: Quality Assurance checklistExplanation

Key Objectives Self-administered partner reporting on the quality, principles and standards of their Programme

What will be done with the collected data?

To inform training requirements at regional meetings

Items covered - Aflatoun Principles- Programme policy revision

Who do we do this for? - Partners- Secretariat

Who executes? The Secretariat has developed the Quality Assurance checklist. All partner organizations within the network are required to fill out the checklist.

Further description - Partner organizations fill out online questionnaire via the member area- They report on the quality, principles and standards of their programme- Secretariat will aggregate results and use them as input for regional meeting trainings and assessment of the standards for partner selection

Projected timeline (also see planning)

- July – Aug: data collection- Sep: internal reporting- Nov - Dec: feedback results in regional meetings

13

Page 14: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Tool: Secretariat participatory evaluationExplanation

Key Objectives - Independent participatory evaluation of Secretariat efficiency & effectiveness- Evaluation of Secretariat’s facilitation of network

What will be done with the collected data?

- Using network feedback for Secretariat and Network Planning & Improvement

Items covered - Effectiveness & efficiency of Secretariat usage of (financial) resources- Quality of communication by Secretariat, communication throughout Network- Evaluation of decision making procedures and level of participation

Who do we do this for? - All stakeholders Aflatoun

Who executes? The development, execution and reporting of the evaluation will be done by an independent (development) consultancy. Throughout the process, partner organizations, other stakeholders and the Secretariat will be consulted.

Further description - The Secretariat develops Terms of Reference (TORs) in conjunction with the network- Methodology for Secretariat participatory evaluation chosen & developed by consultancy- The evaluation cycle will be done every calendar year.

Projected timeline (also see planning)

- Aug: TORs development- Oct - Dec: Projected evaluation period- Mar: Reporting in Annual Report

2.4 Quality Assurance

14

Page 15: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Aflatoun Quality assurance and Impact assessment System

1.Strategy overview – Principles and structure2.Quality Assurance3.Output Tracking & Outcome Assessment4.Impact Assessment5.Planning

15

Page 16: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Tool: Student Surveys & partner databaseExplanation

Key Objectives - Network wide data collection and analysis on output and outcomes of the programmes in the network- Outcome assessment of programme at students’ level- Collection of programme output information at partner, school and class/teacher level- Development of a database of social & economic indicators of the programme

What will be done with the collected data?

- Collected data will be key input for annual programme-reporting publication: “Children of Change”- The indicator database will be improved in years to follow and used as input for impact assessment

Items Covered - Level of implementation of Aflatoun core concept & programme on ground- Size and numbers of stakeholders of partner programmes - level to which students develop expected knowledge, attitudes, skills and behaviour on the basis of the curriculum

Who do we do this for? - Donors- implementing partners- Secretariat

Who executes? • In cooperation with the Impact Assessment Task force, the AQIS committee and the broader partner network, the Secretariat develops student surveys, and partner, school and class surveys.• All implementing partners fill in the partner survey and will collect data at the school level. After the completion of at least 1 year’s programme cycle, partners are asked to collect data at the class and student level. The database is developed in conjunction with our Indian partner MelJol (India) and Tata Consultancy Services India (pro bono).• The Secretariat aggregates data and publishes “Children & Change”

Further description - Test of students’ knowledge, understanding, attitude, behavior, via surveys based on Aflatoun’s curriculum objectives- Surveys currently under developed by survey development consultant- A post-programme student survey will be implemented network wide all partners, all levels- A pre-programme student survey is optional- Partner Database software is developed by TCS India – commissioned by MelJol and the Secretariat- Data collection CD package development including database, surveys and manuals for partner usage- The CD package will also include a SROI web tool (see next two slides)

Projected timeline (also see

planning)

- May: Secretariat finds educational specialist - May: finalization core competency overview of Knowledge, Attitudes, Skills, Behaviours as expected on basis curriculum- July – Aug: questionnaire development- July– Oct: Partner Database software development- Nov – Dec: Presentation of surveys, CD package and manuals at regional meetings- Jan – Feb 2009: data collection- Mar - April 2009: publication “Children & Change” (ed. 2009)

3.1 Output Tracking & Outcome Assessment

16

Page 17: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Tool: SROI web toolExplanation

Key Objectives Understanding and showcasing the Social Return on Investment (SROI) of the Aflatoun programmes in the network.

What will be done with the collected data?

- Collected data will be key input for annual programme-reporting publication: “Children of Change”- The indicator database will be improved in years to follow and used as input for impact assessment

Items covered - Measurable & monetisable programme indicators for social and economic impact- Implementing partner and Secretariat effectiveness

Who do we do this for? - Advocacy tool for partners in the network and the Secretariat in showing the power of the Aflatoun programme

Who executes? The web tool assisting the SROI analysis is currently under development by a social venture of several Dutch foundations. We aim for full partner participation when the web tool is ready.

Further description - Secretariat has developed basic outline of “impact map” for stakeholders in programmes (see Annex 3 in “Children & Change”, ed. 2008)- A SROI web tool, tailored to the demands of the Aflatoun network is currently under development. The expected date of delivery: 3rd/4th quarter 2008- The greater part of data for the SROI analysis provided by the web tool will be collected with student, partner, school and class surveys (see previous page)- The web tool provides “impact maps” per programme reporting on the Social Return on Investment- These “impact maps” will be part of the annual publication of “Children & Change”

Projected timeline (also see

planning)

- July – Briefing of web developer- July – November development web tool- Nov-Dec – Presentation & testing web tool at regional meetings- Jan-Feb ‘09 – Data collection at partner level- March ‘09 – publication “Children & Change” (ed. 2009)-Disclaimer: substantial changes dependent on progress of developer web tool

3.2 Output Tracking & Outcome Assessment

Page 18: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Children’s surveys

School output survey

Partner output survey

Class output survey

Partner app. system

Elements of outcome & output data-collection:

To be developed with MelJol (India-partner)

Two sets of pre- and post-surveys, for workbooks 1-3 and 4-8

Output related questions to understand scope & outreach of programme

SROI web tool

Data-collection CD

Partner SROI

question list

Selection of data for SROI web tool comes from partner application system

Some of questions in children’s survey are taken from SROI web tool

Help & methodology documentation for

partners

Included in CD package

Including: templates questionnaires, help documents, country database software

(inclusion SROI web tool on data collection CD depends on agreed timelines with developers SROI web tool)

18

Output related questions to understand scope & outreach of programme

Output related questions to understand scope & outreach of programme

Page 19: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Tool: Process Review & Reflection (R&R)Explanation

Key Objectives - Review & reflection of partner programme’s effectiveness and efficiency against background of Aflatoun’s strategic objectives

What will be done with the collected data?

- Usage of results to progress network learning & programme development- Collected data will be considered as input for annual programme-reporting publication: “Children of Change”

Items covered Communication, policy, participation and decision making processes, at the various stakeholder levels within partner programmes

Who do we do this for? - implementing partner organisations- Secretariat- (Funders)

Who executes? - Secretariat Programme Managers, researchers, PhD students in social work/sciences, MBAs- in conjunction with implementing programme team

Further description - Process Review & Reflection proforma development, taking into account social & cultural contextual differences- 2-3 week top to bottom analysis of processes within partner programme – programme wide- analysis of effectiveness and efficiency of processes in realisation of Aflatoun’s mission and vision

Projected timeline (also see

planning)

- 3rd Quarter 2008: development of R&R proforma- 4th Quarter 2008: selection of researchers- 1st Quarter 2009: agreement on scope project

- 2 project with implementing partner organisations in 2009- 5 projects with implementing partner organisations in 2010 (regionally spread)- 10 projects with implementing partner organisations in 2011 (regionally spread)

3.3 Output Tracking & Outcome Assessment

19

Page 20: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Tool: Action Participatory ResearchExplanation

Key Objectives - Qualitative study of quality & development of concept of programme on the ground- Concept development- Social and economic indicator database development

What will be done with the collected data?

- Usage of results to progress network learning & programme development- Collected data will be considered as input for annual programme-reporting publication: “Children of Change”- The indicator database will be improved in years to follow and used as input for impact assessment

Items covered - Qualitative study on effectiveness of each of the 5 Aflatoun core elements- Qualitative study on effectiveness of each of the 8 Aflatoun workbooks- qualitative study on effectiveness of programme implementation tools (e.g. piggy bank, bank accounts, Aflatoun clubs)

Who do we do this for? - implementing partners- secretariat- (funders)

Who executes? - independent researchers - Aflatoun programme managers- in conjunction with implementing partners

Further description - Understanding the quality of the concept and investigating avenues for further concept development- Research happening in 1-2-1 with Process Review & Reflection (see previous page)- Focus group discussions, individual surveying, class ledger analysis- TBD

Projected timeline (also see

planning)

- 3rd Quarter 2008: development of research proforma- 4th Quarter 2008: selection of researchers- 1st Quarter 2009: agreement on scope project

- 2 project with implementing partner organisations in 2009- 5 projects with implementing partner organisations in 2010 (regionally spread)- 10 projects with implementing partner organisations in 2011 (regionally spread)

3.4 Output Tracking & Outcome Assessment

20

Page 21: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Tool: Focus Group DiscussionsExplanation

Key Objectives Identifying qualitative, as well as unforeseen, results and outcomes of the programme.

What will be done with the collected data?

Usage of results to progress network learning & programme development- Collected data will be considered as input for annual programme-reporting publication: “Children of Change”- The indicator database will be improved in years to follow and used as input for impact assessment

Who do we do this for? • Partners• Researchers/Academics• Donors

Who executes? - Independent researchers - Aflatoun programme managers- In conjunction with implementing partners

Further description Guidelines for conducting focus group discussion will be elaborated so that partners can collect qualitative data on the programme for external purposes and also for internal learning. The information gathered from the focus group discussions will also allow the Secretariat and evaluation researchers to refine quantitative tools for output and outcome evaluation.

Projected timeline (also see planning)

3.5 Output Tracking & Outcome Assessment

21

Page 22: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Tool: Stories of changeExplanation

Key Objectives Understanding the outcome/impact of the programme on individual lives and communities

What will be done with the collected data?

- Collected stories will be used for the Aflatoun website, publications and other promotional material- Collected stories will be considered as input for annual programme-reporting publication: “Children of Change”- Usage of results to progress network learning & programme development

Items covered - Short stories of change for different stakeholders in the programme

Who do we do this for? • Global Stakeholders• Donors

Who executes? - Partner organizations collect stories and observations of occurring changes with students, teachers, and others within the programme- The Secretariat collects and publishes (a selection of) these stories

Further description Guidelines for collecting real life stories and anecdotes are being developed. These stories and anecdotes aim to share snapshots of how the programme may have a specific impact on a child, class, school, community. They emphasize the description of situations that exemplify social and economic empowerment, showcasing how the mission of inspiring children to be change makers can be achieved.

Projected timeline (also see planning)

3.6 Output Tracking & Outcome Assessment

22

Page 23: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Aflatoun Quality assurance and Impact assessment System

1.Strategy overview – Principles and structure2.Quality Assurance3.Output Tracking & Outcome Assessment4.Impact Assessment5.Planning

23

Page 24: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Tool: Randomized evaluationsExplanation

Key Objectives Rigorous proof of Aflatoun concept in different social, cultural contexts

What will be done with the collected data?

- analyzed research data will be presented in annual publication “Children & Change”- analyzed research data will be disseminated throughout network meetings and via international stakeholder networks for optimal insight in CSFE concept

Items covered - Controlled evaluation of programme elements- Academic proof of the Aflatoun concept- usage of social and economic indicators developed with class questionnaires, SROI and action participatory research

Who do we do this for? Policy makers, Donors, (potential) Partners

Who executes? - Depending on grant developments: Innovations for Poverty Action (Dean Karlan)- in close and intensive conjunction with partner programmes

Further description In cooperation with Dean Karlan, Professor of Economics at Yale University and director of Innovations for Poverty Action (IPA), Aflatoun desires to start two or more randomized evaluation trials at different programmes in our network. Randomized evaluation is placed amongst the most rigorous approaches towards measuring impact of social and development programmes.

Projected timelines (also see Planning)

- Involvement of randomized evaluations in strategy discussion AQIS at international stakeholder meeting 2009- 2010-2012 randomized evaluations with implementing partner programmes

4.1 Impact Assessment

24

Page 25: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Tool: Longitudinal studiesExplanation

Key Objectives Understanding the longer term social and economic impact of the Aflatoun programme in different social, cultural and economic contexts.

What will be done with the collected data?

-The complete approach of impact assessment will be implemented throughout the network

Items covered - Research for available & appropriate methodologies for impact assessment- study of approaches in relevant fields (e.g. microfinance, education)

Who do we do this for? Partners, Donors, Academics

Who executes? - Secretariat- Impact Assessment Taskforce & AQIS committee- broader Aflatoun network

Further description Aflatoun has consulted various researchers in the field about the possibility of performing longitudinal studies in our programmes. These projects would help significantly in understanding the deeper changes that occur in the lives of our target group. The costs of these projects are high, and require a long term commitment of Aflatoun, its partner(s) and the concerning research institution. The network discussions on these studies are planned to start throughout 2009.

Projected Timeline 2008- (ongoing)2009 – at the next international stakeholder meeting progress of research will be presented

4.2 Impact Assessment

25

Page 26: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

Aflatoun Quality assurance and Impact assessment System

1.Strategy overview - introduction2.Quality Assurance3.Output Tracking & Outcome Assessment4.Impact Assessment5.Planning

26

Page 27: Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

27

5.1 (Suggested) planning 2008-2013 all AQIS issues

1st Quarter 2008

2nd Quarter 2008

3rd Quarter 2008

4th quarter 2008 1st Quarter 2009 2nd Quarter 2009

3rd Quarter 2009

4th Quarter 2009

2010 2011 2012 2013

A3 Self-evaluation questionnaire (QA) LYNX Participation

Developm QA web tool

Developm QA web tool

Data collection

Feedback in trainings Inclusion in Children & Change

Updating web tool

Data collection

Feedback in trainings

Repeat 2009 cycle

Repeat 2009 cycle

Repeat 2009 cycle

Repeat 2009 cycle

Evaluation Evaluation

Report Report

Survey develop

- Survey develop

Data collection

-Task Force meets

Data analysis

DB develop optional pre-survey implement

Publication of Children & Change

- DB develop

Data collection

Data analysis

Publication C&C

C3 Process review & reflection (R&R) LYNX Participation

-TOR develop - Selection researchers

- Agreement projects 2 projects start 2 project reports

-5 new projects

- 10 new projects

- 10 new projects

- 10 new projects

- C&C - C&C - C&C - C&C

- 5 new projects

- 10 new projects

- 10 new projects

- 10 new projects

D1 Randomized evaluation LYNX Participation

Application for funding

Decision on funding Preparation Preparation Start projects Continue Continue Reporting + start 2 new projects

Continue Continue

D2 Longitudinal studies LYNX Participation TBD

Scoping Scoping Scoping Start project

TBD TBD TBD

LYNX Participation to be clearly delineated during Aflatoun's African Regional Meeting Nov. 24-28 and subject to funding resrouces according to assessment in November 2008

Repeat 2009 cycle

Repeat 2009 cycle

Repeat 2009 cycle

- Agreement projects 2 projects start 2 project reports

Repeat 2009 cycle

Repeat 2009 cycle

Repeat 2009 cycle

Testing Web tool at regional mtngs

Repeat 2009 cycle

Repeat 2009 cycle

C4 Participatory Action Research LYNX Participation

-TOR develop - Selection researchers

Suveys & DB presentation at regional meetings

Repeat 2009 cycle

Repeat 2009 cycle

C2 SROI - LYNX Participation

SROI web tool develop

SROI web tool develop

Web tool finish & test

Web tool introd. at regional meetings

Changes to web tool

Evaluation of followed approach

Repeat 2009 cycle

Repeat 2009 cycle

C1 Class Surveys & Partner database -LYNX Participation

Presentation Surveys & DB at regional meetings

Changes to surveys, changes to DB

A4 Aflatoun Network evaluates Secretariat effectiveness & efficiency (LYNX)

Set up TOR & selection consultancy

Inclusion in Annual Report