manual de evaluación participativa

239
Participatory Evaluation: A user guide and Manual Specially prepared fCambodia Global Action and Samaritan's Purse International 1 © Dr. Ravi I. Jayakaran, QPI-GMS®, June 2007 Prepared for the CHE program team working in 76 villages in Cambodia as part of Cambodia Global Action.Made available through special funding support from Samaritan's Purse International Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Upload: geovanny-santamaria

Post on 08-May-2015

217 views

Category:

Technology


1 download

TRANSCRIPT

Page 1: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

1

© Dr. Ravi I. Jayakaran, QPI-GMS®, June 2007 Prepared for the CHE program team working in 76 villages in Cambodia as part of Cambodia Global Action.Made available through special funding support from Samaritan's Purse International

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 2: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

2

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 3: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

3

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 4: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

4

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 5: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

5

Photographs in this manual are by author unless otherwise specified.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 6: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

6

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 7: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

7

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 8: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

8

This manual is dedicated to the poor and marginalized in Cambodia, and to the task of equipping CGA staff to become good Participatory Evaluators who

will continue to seek to improve the performance of their programs and be committed to Continueous Quality Improvement (CQI).

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 9: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

9

A user Guide and Manuall

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 10: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

10

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Photo: Joanna Bryden

Page 11: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

11

Chief Consultant: Strategy & Development Programming

Quality Professional Interfaces-GMS® Cambodia

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 12: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

12

Index:

Chapter Subject Page number

1. Introduction

2. Cleaning up the design Matrix

3. Background and Principles of Participation

4. The Uniqueness of Qualitative information

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 13: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

135. Types of groups to be interviewed and documentation

6. Using the Manual

7. Scheduling & Evaluation design

8. General information at village level

9. Purpose of Evaluating Programs

10. Stage when Evaluation should be done

11. Strategy for Evaluation (How)

12. Preparing for an evaluation:

13. Stakeholder analysis

14. Difference between Monitoring & Evaluation

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 14: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

1415. Annual reviews and Ministry Reviews

16. CQI and Strategic Planning

17. Determining what to do

18. Planning ways to do it

19. Designing a tailor-made Evaluation

20. Various Participatory Exercises

21. o Country profile

22. o Impact-1: Community health improved

23. o Impact-2: Family Food security status improved

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 15: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

1524. o Impact-3: Increased capacity to manage their own lives

25. o Impact-4: Community infrastructure provided

26. o Expenditure analysis – at Macro level

27. o Uncertainity analysis – at Macro level

28. o Problem analaysis – at Macro level

29. o Livelihood analysis – at Macro level

30. o Wholistic Worldview Analysis (WWVA)

31. o Community Attitude to CHE program

32. o Community Attitude to Christ

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 16: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

1633. o Community Attitude to the Change agent

Index (contd…)

Chapter Subject Page number

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 17: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

1734. o Grouping of villages according to their performance

35. o Key stakeholders in the development process

36. o Stakehoder analysis (in a successful village)

37. o Stakehoder analysis (in a partly successful village)

38. o Stakehoder analysis (in a village with low results)

39. o Uncertainity Analysis at village level

40. o Problem Analysis at village level

41. o Livelihood Analysis at village level

42. o Expenditure analysis at village level

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 18: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

1843. o Community level program inventory

44. The Default Priority Profile - DPP

45. The Value Change Index

46. EEIRS (Efficiency, Effecitveness,Impact,Relevance, Surstainability) test for the overall performance of the organization

47. Acronyms used in this manual

48. Conclusion

49. Notes on the TST

50. Additional reading

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 19: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

19

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 20: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

20

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 21: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

211. Introduction:

Samaritan’s Purse International Relief (SP) is Christian Non-Governmental Organization that has been providing relief to people in need around the world since 1970. For the past 23 years, SP has helped meet the needs of victims of war, poverty, natural disasters, disease and famine. SP responds quickly, appropriately and has been active in over 100 countries worldwide. Although largely donor funded, SPIR has successfully partnered with donors such as USAID, CIDA, DFID, UNICEF, WFP and UNHCR.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 22: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

22For almost 10 years, SP has been helping to provide financial and human resources to a variety of local and international NGOs operating in Cambodia. Most recently, SP has ventured into a partnership with Cambodia Global Action (CGA) to grow their CHE programming with animal husbandry and village credit programmes. The Seeds of Hope programme offers seed resources for animal banks, impact monitoring and CHE trainings to over 80 CHE committees throughout Cambodia. It is SP’s long term desire to open this programme to other CHE-based projects being managed by other LNGOs in Cambodia. This relationship with CGA is considered a pilot of more expansive programmes for others working with the CHE methodology.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 23: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

23CGA is a local Christian NGO with CHE programming for over 5 years in 76 villages across 9 program units with 16 sub project units at a commune level.

1) Kompong Speu 1. Kandal Dom Commune, Chbaa Mon District2. Mohasaing Commune, Phnom Srouch District

2) Kandal (Khien Svay) 3. Dey Eth Commune, Khien Svay District4. Bantey Dek Commune, Khien Svay District

3) Kandal (Saang) 5. Svay Rolum Commune, Saan District6. Kos Anlong Chin Commune, Saang District

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 24: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

244) Kampong Cham 7. Teuk Chroow Commune, Dambae District

8. Chong Cheach Commune, Dambae District9. Nean Toet Commune, Dambae District

5) Phnom Penh (Dangko) 10. Trapean Kransang Commune, Dangkor6) Kampot 11. Andong Khmae Commune, Kampong Bay

12. Meak Prang Commune, Kampot District7) Siem Reap (Pouk) 13. Keo Por Commune, Pouk District8) Takeo (Bati) 14. Trapeang Krasang Commune, Bati District9) Stung Treng 15. Samahki Commune, Stung Treng District

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 25: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

25

This Manual has been Tailor-made for the CHE program of CGA but has principles that can be adopted in other programs too.

2.Cleaning up the deign Matrix (Making the Log frame Evaluation friendly):

Before carrying out an Evaluation, one must ensure that the design Matirix (PDM-Project Design Matrix) that forms part of the Log frame for the project's programs is properly streamlined so that activities result in Out-puts of significance, and that these in turn end up together having an Impact. A series of 3 or 4 Impacts then lead

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 26: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

26in the direction of enabling the Project or Program achieving its Objectives or Goals. Evaluations are carried out at the Impact level, hence, if the Design Matrix isn't properly integrated, it will be difficult to Evaluate or attribute success to any particular program. In reviewing the design of the CHE program of CGA, we found that there was some misalignment of the activities, Outputs and impacts. Time was therefore taken to re-align these so that they fitted more correctly into a logical sequence. The details of the revised Matrix are seen below:

Impact I: Improved Community Health AchievementOutput I.1: Improved Primary Healthcare

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 27: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

27

Activity I.1.1: Community Health Screening

Output I.2: Improved Water/Sanitation

Activity I.2.1: LatrinesActivity I.2.2: WellsActivity I.2.3: Soap DistributionActivity I.2.4: Water Filters

Output I.3: Improved Nutrition

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 28: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

28

Activity I.3.1: Home GardenActivity I.3.2: Rice BankActivity I.3.3: Rice Storage

Output I.4: Improved Environment

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 29: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

29

Activity I.4.1: Organic Pig PenActivity I.4.2: Earth Worm RaisingActivity I.4.3: Garbage ManagementActivity I.4.4: Compost PitActivity I.4.5: Fruit Tree NurseriesActivity I.4.6: Bio Gas

Activity I.4.7: Natural Pesticide

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 30: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

30The same principle was applied to all the Impact areas as follow:

Impact II: Family Food Security Increased Achievement Output II.1: Micro-Enterprises Increased

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 31: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

31Activity II.1.1: CreditActivity II.1.2: Rice Straw MushroomActivity II.1.3: Soybean SeedActivity II.1.4: Self Help GroupActivity II.1.5: Food ProcessingActivity II.1.6: Fish Ponds

Output II.2: Animal Raising IncreasedActivity II.2.1: Chicken RaisingActivity II.2.2: Cow BankActivity II.2.3: Goat BankActivity II.2.4: Pig Bank

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 32: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

32

Impact III: Increased Capacity to Manage Their Lives Achievement Output III.1: Improved Spiritual Lives

Activity III.1.1: Video EventsActivity III.1.2: Education

Output III.2: Improved Basic EducationActivity III.2.1: Village Livestock Agent Projects Activity III.2.2 Literacy ClassActivity III.2.3: TrainingActivity III.2.4: Family Competition

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 33: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

33 Output III.3: Improved Community Structure

Activity III.3.1: VDCs, PMCs, CHEs and CHE ModelImpact IV: Community Infrastructure and Relief Provided Achievements Output IV.1: Community Infrastructure Created

Activity IV.1.1: Road Repairing (Food for Work)Activity IV.1.2: Water Reservoir (Food for Work)Activity IV.1.3: School Construction with Community

ParticipationActivity IV.1.4: Sewage System with Community

Participation

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 34: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

34

Once this sorting out was done, it became relatively easy to decide what the framework for the Participatory Evaluation should be.

3. Back ground and Principles of Participation:

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 35: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

35

The TST (which has been described in detail in the briefing notes in Chapter 46) is used a great deal now in the Greater Mekong Sub-region for Participatory development programs. One of the biggest advantages with this technique is that it allows for discussion of information that is otherwise sensitive to handle. Besides this, while using the seeds and grouping them to depict information, there is the creation of a visual that tremendously enhances the quality of discussions. Also because of their ‘non-threatening nature’ the seeds allow group members to feel free to express themselves openly and frankly. It is important while facilitating the group discussions to ensure that all members in the group get a fairly equal opportunity to share their views and that they are not dominated over by any group members and

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 36: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

36coerced into agreeing to a ‘uni-polar’ view. This therefore means that the group should be of an optimum size of 7-9 members (discussed again later) for best results.

Facilitation is an art. In fact it is a way of life! Care must be taken to ensure that the core evaluation team members get ample input to develop their capacities to become good facilitators. This applies also to those who will be added to the core team in later months. All of them must go through the basic training and go through the pre requisite orientation to ensure that they do not become ‘manipulators’ of opinions, but rather become good facilitators who are able to bring out from the

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 37: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

37group they are facilitating, their true and real opinions. The purpose of Participatory Evaluation is to get an honest picture of what we are doing and the type of impact we are having…not to fool ourselves into believing that everything is perfect!!

While facilitating, ensure that there is adequate time available for the exercise. This involves planning well in advance and ensuring that all members of the group are comfortable. It is often a good idea to have some fruit available for community participants especially as the day moves on close to lunch time or toward the evening. Before starting the facilitator ensures that at least one person has been given the task of documentation of all the discussions. Working in teams is important

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 38: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

38for this reason. After the group has been comfortably seated, introduce the purpose of the exercise, the methodology and then proceed. All other prerequisites by way of materials required, and preparation are mentioned in the notes for each exercise.

4. The uniqueness of Qualitative information:

Qualitative in formation has a different mandate from Quantitative information. This must always be remembered by the Evaluater. While preparing the report one must capture the uniqueness of the context that will emerge when facilitation is done well.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 39: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

39Each site must have the same number of exercises conducted for uniformity. Unlike quantitative information, the outcome of the exercises will be a narrative document with the visuals of each exercise in it. Comparing between sites is therefore is not so much a matter of something being ‘better than’ or ‘worse than’ but a matter of looking for linkages and learning from the composite information.

Qualitative information allows for obtaining width as well as depth to the information and this is what gives it its richness. To begin with, the principle is to ‘cast the net wide’ and get as much width from the information as possible. When something is seen to be significant, it is good to go into depth and probe as much as possible for

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 40: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

40hints of reasons and causes. In some cases it may even be possible to carry out case studies of people who are seriously affected. This often throws light on reality in a most unusual way. For example, we may facilitate through awareness and capacity building, ‘sexual negotiation skills’, but the whole issue of condom availability in the village may be the biggest stumbling block. Again, even if condoms are available in the village, who they are available with may become an issue! Again in conservative society, it may be difficult for girls to ask questions of clarification on issues that are not immediately clear, and the findings may be to just go ahead and list some of the questions that ‘normally arise’ and answer them.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 41: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

415. Types of groups to be interviewed & Documentation:

The Evaluation process involves interviewing individuals, and small homogenous group such as a government department where only one person may function as a spokesperson or with a small group where everyone is free to participate. The technical term for the first type is ‘Key informant interview’ or KII. The second type is referred to as a ‘semi- structured interview’ or SSI, and the third one is referred to as a focused Group discussion. The exercises designed for this manual are all specifically designed for Focused Group Discussions (FGDs), but they can also be used for the SSI and the KII. The dynamics of group interaction allows for

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 42: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

42triangulation (correction and modification by others in the group) of the information.However in both KII and SSI there is no scope for Traingulation. The ideal size of a FGD is 7 to 9 members. Care should be taken to get a random representation along with reasonable gender balance and age balance (when relevant). Those selected must have a good idea about the topic so that they can all participate and contribute to the discussion. In the design of each exercise, it has been recommended (where relevant) to also discuss and share the findings of the FGD with a larger group to ensure that they also share the same views. When this is not the case, a note should be made in the document of the changes that were suggested. Also, ensure that the original flip chart is left behind with the members of

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 43: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

43the FGD or some responsible person at the site. This means that the documenter must copy the final visual also as part of her/his documentation.

Documentation: Each exercise has a code that begins with ‘PE’ followed by a number. During the Pilot assessment, orientation and training, the core team will be shown how to use this exercise code and also determine some guiding questions that would be useful to facilitate the FGD. While the facilitator is facilitating the group discussion, the documenter will note (as bullet points) all the important points that emerge. At the end of the discussion he/she will also copy the visual. At the end of the day, each team will look at the exercises that it has completed and read through

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 44: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

44the documentation, highlighting the important points. This also helps edit out points that have repeated themselves in the course of documentation. The highlighted points are synthesized into a summary that is about 1-2 paragraphs in length. To summarize, each exercise will have the following:

The Code with guiding questions A copy of the visual A transcript of the discussions with important points highlighted A summary of the important points.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 45: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

45The summaries (written in sequence) will constitute the site summary. The original copy of the visuals and code with transcript (as mentioned above) will be preserved in a box file at the main office for reference when required. As mentioned earlier, the original flip chart on which discussions were carried out and which has the original ‘visual’ must be left behind at the site with the group interviewed.

6. Using the Manual:

The following are the various parts of each exercise, as described in the succeeding chapters. Each chapter uses the same format. This has specially been written this

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 46: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

46way so that it is extremly user friendly and each exercise has all the details of what needs to be done. Every facilitator must study this clearly and understand the design.

7. Scheduling and Evaluation design:

The following exercises will require between 2-3 full days for one team to carry out in a village. The ideal team will consist of three members, whose roles will be interchangeable:

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 47: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

471. A team leader: to help with logistics, encourage group members to

participate, counter domination, and also prepare for the next exercise so that there isn’t any wastage of time in transitioning from one exercise to the other. The team leader will also be responsible for 'Filtering off'and dominant people in the group and for documenting case studies when they emerge.

2. A Facilitator: to facilitate the exercises as planned

3. A documenter: to document the details of the discussions

Alternately, if the PE team members work in teams of 5 and are located in the same place during the assessment, they can have one common team leader for two teams

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 48: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

48of two each. Each of the teams will have a facilitator and a documenter. The schedule for carrying out the exercises and the groups with which it should be carried out should be planned out in advance, before going to the village.

After completing the exercises the following is the Filing profile for individual exercise reports:

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 49: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

49

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 50: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

50

Shown above is the format used for the recording of the details of each exercise. These original documents should be stored in a box file at the CHE/CGA office in Phnom Penh. The code mentioned below in the summary as well as the individual

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 51: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

51exercise documentation enables the reference of the original papers when additional or detail information is sought.

Coding pattern:

PE-01

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Participatory Evaluation

Exercise number

Page 52: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

528.General Information (at Village Level): At each site, obtain the following information that is relevant to the site and will help the team understand the details of what is happening in the process at the particular site. The following format shows how to fill the information and the last column also shows where the information can be obtained:

Details of information Source

Name of the Village staff

Date of starting the Program staff

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 53: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

53

Type of School ( Primary / Secondary) Village chief

Total land area of the village Village chief

Population density = Popn./ land area in KM2 Calculate

Infant mortality rate Calculate

Vaccination levels Use TST

Ratio of Doctors to population Calculate

Average rice production per hectare Village chief

Average rainfall in the area School Atlas of Cambodia (P.12)

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 54: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

54

Poverty rate (Neik Krao) RFSA

Poverty rate (Neik Tual Krao) RFSA

% of population with access to clean water TST

% of children in school TST

% of community migrating out every year TST

Visible display of acknowledgement of donor contribution to the Program: (Y/N)Note: Photo copy this template and get information for each village where Participatory Evaluation is being done.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 55: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

55

The general information on each village will enable the Evaluators to see what progress has been made in the village over the years and also enable the identification of areas which are still below standard.

9.Purpose of Evaluating Programs: If there is one thing that gets neglected in the project and Program cycle, it is the Evaluation of the impact of the program at periodic intervals. When a program is launched, it is done on the basis of some assumptions. These assumptions of imact are assumed to be the result of a group of outputs, which in turn are the result of a group of activities. When the program is actually put into place, some of these assumptions may not work out, or the impact

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 56: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

56may turn out to be different from what was intended. If the program is however evaluated every two or three years, the actual impact can be determined and this will help the management detrmine if the startegy employed was appropriate or not. Accordingly, corrections can be made in the program for bringing about improvement, dropping activities that are not relevant, or introducing new programs that may be required for corrective action. An evaluation is an excellent opportunity to look back at what has been achieved against the planned objective or goal of the program. It enables the Management of the program to see what progress has been achieved and celebrate it. The next section will talk about the time and stage when various evaluations are possible in greater detail.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 57: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

57

10.Stage when Evaluation should be done: There are different stages when an evaluation can be done. Smaller, and more focussed evaluations can be done at various stages of the Project or Program cycle as seen in the diagram.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 58: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

58

EVALUATIONS

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 59: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

59

For this, various aspects of the program cycle such as needs assessment, Planning, Implementation and monitoring system in place are evaluated. This type of Evaluation is manily done when there are implications of a new methodology or a program that requires wider implementation, and for which the initial 'pilot' program is to serve as a model. These evaluations are shorter in duration and may require people who are specialists on the program component that is being evaluated.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 60: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

60

PHASE OUT

3YRS

EOP-EProgram Life cycle and Evaluation

SEED PHASE 1ST 5 YEAR DESIGN

2YRS 5 YRS

QUALITATIVE(PLA)

QUANTITATIVE(30 CLUSTER HHS

SECONDARYDATA

E-1

1ST 5 YEAR DESIGN 2ND 5-YEAR DESIGN

5 YRS 5 YRS

E-2 E-4

E-1

E-3

The third evaluation helps plan the phase out design of the project.

Dr.Ravi Jayakaran: WV Cambodia

E-2 E-5

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 61: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

61

For larger, long term projects and programs, it is necessary to build in Evaluations at the end of each phase. Ideally, evaluations should be done at a 2-3 year interval, which means that the longer phases will get an additional mid-term evaluation also. Often, with projects and programs, the weakness is that at the time of writing the proposal, not provision was made for evaluations. They then try to look desperately for a budget when they require and evaluation and it often ends up as an unplanned, shallow and shoddy affair. Ensure that you budget for evaluations at regular intervals when you design your program. This will ensure that planning is also done with evaluations in mind, namely that program people will always be conscious that the

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 62: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

62program will be evaluated for impact. Evaluation reports when positive can open up the door for new and additional funding.

11.Strategy for evaluations: The Program design Matrix is the framework around which the evaluation is conducted. This therefore requires for the design to be well thought through and for activities, outputs and impacts to be integrated. In an earlier chapter we discussed that this needs to be 'cleaned up' if not done already before starting the evaluation. The principle then is to start with the Macro and then move down to the output level. As in the case of the CHE PDM, there are 4 main impact areas. The first step therefore is to see what impact has been achieved under each

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 63: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

63impact area, namely improving health, improving household food secuity, increasing capacity to manage their lives and creating new infrastructure. Changes in the lives of the community in these areas is explored and studied. Evaluations can be done based on quantitative data or qualitative data or a combination of both. For quantitative data to be of use, one must however have details in the form of a 'baseline' carried out at the start of the project. This manual details how to use qualitative techniques for carrying out an evaluation.

12.Preparing for an Evaluation:

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 64: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

64In preparing for an evaluation, one has to determine what aspects of the impact of the project are to be evaluated and by what means. For some of the aspects, there will be quantitative data available in terms of the numbers of deliverables that ahave already been generated by the program such as wells, or water filters, or latrines etcetra. This data should be documented for each of the activities and outputs under each impact area. These are available for review in the Appendix that accompanies this document. These hard numbers will show how progress has been made over the years and will especially be useful for the donor to be certain that their money was used wisely in producing the outputs that were planned. The evaluation doesn't stop here, but goes further to also measure how those outputs which were

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 65: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

65generated as part of the project activity impacted the lives of the community that was part of the target of the community. Staff teams are made and then they go along with external representatives to places other than where they normally work, to carry out the evaluation. Where possible, case studies are also developed of examples of both success and failure, so that they can be valuable lessons learned. After the evaluation is scheduled and starts, the external evaluator randomly visits some of the sites, to see how objectively the evaluators are doing their work. The exposure to other areas of work within the project also gives the staff a chance to see how their colleagues are working. There is cross fertilization of perspectives and they also

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 66: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

66draw new insights and lessons from the whole process. At the end of the evaluation, there is opportunity to debrief and share ideas and lessons learned.

13.Stakeholder analysis:

Another pre-requesisite for the evaluation is to carry out a 'stakeholder analysis'. This consists of finding out who the potential stakeholders are for the information that is likely to emerge from the evaluation.The stakeholder analysis is done using the TST and determining details with a focus group that consists of members representing the donors, the Board, the Management team and if possible one of the

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 67: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

67community leaders from the target area. The following page shos an example of how a stakeholder analysis looks. In the process of this Analysis we try to assess what each stakeholder's 'stake'is the outcome of the evaluation. We also find out what their principle interest is in the evaluation and this enables us to determine the framework for the evaluation and identify the issues that need to be studied.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 68: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

68

•For self assessment and for improving impact and program modification

Staff

• For improving further collaborationOther partners

•As an exercise to empower them and for greater accountability

Community benefiting from the program

•For ensuring there is good impact and for CQI (Continuous Quality improvement

Management team

•To ensure that the program is progressing well

Board of Directors

• To assess how well Fund was given

•To consider future fundingDonor (existing and potential new ones)

Remarks if anyWhat they are interested in

Key stakeholders

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 69: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

69

At the end of the evaluation, this stakeholder analysis enables us also to detrmine if we have found out all the information we intend to find out about.

14.The difference between Monitoring & Evaluation:

While the words Monitoring & Evaluation are often used together, there is a big difference that exists between the two of them. Monitoring is something that is ofeten done by the staff involved with the program activity and their supervisors. While the supervisor doesn't normally do all the monitoring, he or she should slowly develop a system of keeping track of progress in terms of completion of activities and what has

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 70: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

70been achieved in less measure than was intended. Monitoring is essentially a largely self'done and self assessed activity. Monitoring is also done at the output level, and this may be done at 6 monthly or annual intervals, as compared to the previous one in which activities are monitored at a monthly or quarterly interval. Evaluation, on the other hand is done at a 2-3 year interval when it is actually possible to assess what the impact of the program has been on the lives of the people. Some people also

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 71: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

71

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 72: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

72add an addition dimesion to their assessment by having an 'annual review'. This essentially consists of have a look at all that the program did over the past year and documenting the results of these.

15.Annual Reviews and Ministry reviews:

When there are several ongoing programs and activities continue over several years, it also makes sense to introduce a 'Ministry review' every 5 years or so, to study and recognize what the program is actually busy with, and if parts of it can be changed and reviewed. As with evaluations, It is always good to get an external

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 73: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

73person to facilitate the 'Ministry review', whicle the annual reviews can be done by existing staff with the rpecaution of getting them to review in places other than where they work on a daily basis. Both these exercises should be followed by reflection and correction to programs and activities, and ofcourse, a celebration of achievements.

16.CQI & Strategic Planning:

The annual reviews and the evaluations result in corrective changes for improvement of the focus of the programs and for greater efficiency and impact. These corrective changes facilitate what is known as a process of 'CQI' or Continous Quality Improvement. The Evaluations and the Ministry reviews on the other hand

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 74: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

74feed into the process of Strategic direction planning or Strategic planning and ofetn usher in major changes in the programs, enhancement of activities or changes in the program portfolio so that the organization achieves its goals to a greater extent.

17. Determining what to do:

It is important to determine what to do in advance and schedule this carefully. Good planning will enable the evaluation team to achieve much in its time of evaluating. When an organization experiences evaluations for the first time in its life, and there isn't an evaluation design available in advance, it might be useful to carry out a pilot

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 75: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

75assessment to determine what needs to be done. This method was adopted in the CGA-CHE program that had not had an evaluation in its several years of being in operation. Carrying out the pilot assessment helps in puuting together a 'tailormade'design for the evaluation process.

18.Planning what to do:

After the list of exercises are determined, the evaluation team meets together in preperation and all members are given necessary orientation and or training for carrying out the same before they go to the variuos sites for the evaluation.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 76: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

7619.Designing a tailor-made evaluation:

As described above, the evaluation of the CGA-CHE program was first subject to a pilot assessment, to determine what programs should be assessed, and then based on what was practically possible the evaluation framework and design were tailormade for the needs of the organization. Once established, this can now become the norm for future evaluations, with specific 'additional exercises' being added to this based on information that may be required to write future program proposals or to fulfill donor needs.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 77: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

7720. Various Participatory Evaluation exercises: Based on the pilot assessment, it was determined that some 25 Participatory evaluation exercises be designed for collecting the information that is required by the program and potential donors for future programming and funding. By carrying out these exercises, the program will be able to generate information on various aspects of its work to be able to determine impact and also identify areas which need to be discontinued or modified for better impact. Failure is not something to be ashamed of or worried about. To quote a famour innovater, "Errors are not mistakes, they are innovations gone wrong". Every failure is thus an opportunity for change, modication and improvement!

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 78: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

78

The following are the detailed descriptions of the exercises that can be done in the field for getting needed information. Each exercise has descriptions and an example from actual use in the field:

21.Country profile: This exercise consists of carrying out a macro profile of the country where the program is operating. In generating some of the basic demographic details of the country the Program team mebers learn where to get

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 79: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

79various bits of information. For Cambodia the main sources from where information can be obtained are: The Ministry of Planning, The National Institute of Statistics, The Ministry of Education, Youth and Sports (MoEYS) and from the Atlas of Cambodia and the school Atlas of Cambodia (available at Monument book stores).

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 80: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

80

6/15/2007 Ravi Jayakaran / Participatory Evaluations

1

1:3124Health facility (doctors: Population)

252Total number of High schools

670Total number of Secondary schools

1429Total number of Primary schools

8628Total number of schools

74Population density/ Square KM

35Poverty rate (extreme poor)

79%Literacy rate (15-44 age group)

1400mmAverage rainfall (annual)

1967/hRice produced (Kg /Hectare)

13694Total villages

185Number of Districts

24Number of Provinces

In CHE’s cover areaIn all of Cambodia

Details

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 81: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

81

Generating this information in the format shown above, and matching it with information from the full area where the CGA-CHE program operates, enables the team to see which areas it has been able to achieve a critical concentration presence for 'tipping the balance'in favour of change.

Besides doing this at a countryoffice level, the program should also fill details in at each field centre level. This information also comes in handy for lobbying for change and advocacy with the decision makers for changing the situation in a particular area.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 82: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

82

Code Number: PE- 01 Time required: 20-25 minutes

Name of the exercise: Impact-1: Impact of Improved community health.

Materials required: Flip charts, marker pens, seeds, masking tape.

Preparation: Read the Briefing notes on the TST, and after the focus group of Program staff responsible for this impact has been briefed about what is expected of them, proceed with the exercise.Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 83: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

83

Example:

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 84: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

84

6/15/2007 Ravi Jayakaran / Participatory Evaluations

2

• Some people still don’t have latrines

•Don’t wash hand regularly

•Some don’t have water filters

•Still lack clean water for all

•Children still dirty

•Some people still don’t have a rubbish pit

•Use latrines

•Know how to prevent disease

•Use water filters

•Reduced water Bourne disease

•Multiply knowledge on health

•Clean environment

•Reduced health expenses

•Increased vaccination

•Get ANC (Ante Natal care)

DetailsTSTDetailsTST

Yet to be achievedAchieved already

Macro picture in the 76 villages where CHE works

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 85: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

85

Steps involved: Ask the Focus group to group the seeds together into 2 groups, one depicting the impact that has already taken place and the other showing the impact that is yet to take place. After there is consensus on this, the group is asked to give details of actual and specific impact in terms of new acquisitions, innovations and changes under impact achieved and areas of shortcomings in the second group.

Analysis of the information: In the example above, we find that around 60 % of expected impact has been achieved, while 40 % on the impact is still to be achieved. Details of these provide us information on what can be celebrated and what still needs focused attention for future improvement.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 86: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

86

Code Number: PE- 02 Time required: 15-20 minutes

Name of the exercise: Impact-2: Impact of Improved Houshold foodsecurity status.

Materials required: Flip charts, marker pens, seeds, masking tape.

Preparation: Read the Briefing notes on the TST, and after the focus group of Program staff responsible for this impact has been briefed about what is expected of them, proceed with the exercise.

Example:

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 87: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

87

6/15/2007 Ravi Jayakaran / Participatory Evaluations

3

• Some people are still lazy to start of their own business

•Some people still continue to use the old techniques

•Some people don’t want to have their own home gardens

•Some people are able to establish their own business

•Started using the techniques

•Some people are able to grow vegetables for their families for the whole year

DetailsTSTDetailsTST

Yet to be achievedAchieved already

Macro picture in the 76 villages where CHE works

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 88: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

88

Steps involved: Ask the Focus group to group the seeds together into 2 groups, one depicting the impact that has already taken place and the other showing the impact that is yet to take place. After there is consensus on this, the group is asked to give details of actual and specific impact in terms of new acquisitions, innovations and changes under impact achieved and areas of shortcomings in the second group.

Analysis of the information: In the example above, we find that around 40 % of expected impact has been achieved, while 60 % on the impact is still to be achieved. Details of these provide us information on what can be celebrated and what still

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 89: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

89needs focused attention for future improvement. In Foodsecurity status improvement, it is obvious that a lot of improvement is required.Code Number: PE- 03 Time required: 20-25 minutes

Name of the exercise: Impact-3: Impact of Increased capacity to manage their own lives.

Materials required: Flip charts, marker pens, seeds, masking tape.

Preparation: Read the Briefing notes on the TST, and after the focus group of Program staff responsible for this impact has been briefed about what is expected of them, proceed with the exercise.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 90: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

90Example:

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 91: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

91

6/15/2007 Ravi Jayakaran / Participatory Evaluations

4

•Some members of the community have not got the techniques of SRI

And the improved Veterinary techniques

•Technique of SRI

•Technique of the Veterinarian

•Know How to take care of their own health

•Know how to increase their income

•Know about being Gender sensitive

•Have good management of their own families

DetailsTSTDetailsTST

Yet to be achievedAchieved already

Macro picture in the 76 villages where CHE works

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 92: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

92

Steps involved: Ask the Focus group to group the seeds together into 2 groups, one depicting the impact that has already taken place and the other showing the impact that is yet to take place. After there is consensus on this, the group is asked to give details of actual and specific impact in terms of new acquisitions, innovations and changes under impact achieved and areas of shortcomings in the second group.

Analysis of the information: In the example above, we find that around 60 % of expected impact has been achieved, while 40 % on the impact is still to be achieved. Details of these provide us information on what can be celebrated and what still needs focused attention for future improvement through greater or modified effort.Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 93: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

93

Code Number: PE- 04 Time required: 20-25 minutes

Name of the exercise: Impact-4: Community infrastructure provided.

Materials required: Flip charts, marker pens, seeds, masking tape.

Preparation: Read the Briefing notes on the TST, and after the focus group of Program staff responsible for this impact has been briefed about what is expected of them, proceed with the exercise.

Example:

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 94: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

94

6/15/2007 Ravi Jayakaran / Participatory Evaluations

5

•Community Bridge

•Road

•Big irrigation system

•Many more schools required

•Big sewage system

•Many more Churches required

•Electricity

•Hospital

•Clean water systems for larger populations

•Farms

•New school building

•Church constructed

•Village path developed

•Health centre constructed

•Sewage system developed

•Irrigation system developed

•Pond dug

•Small farm established

DetailsTSTDetailsTST

Yet to be achievedAchieved already

Macro picture in the 76 villages where CHE works

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 95: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

95

Steps involved: Ask the Focus group to group the seeds together into 2 groups, one depicting the impact that has already taken place and the other showing the impact that is yet to take place. After there is consensus on this, the group is asked to give details of actual and specific impact in terms of new acquisitions, innovations and changes under impact achieved and areas of shortcomings in the second group.

Analysis of the information: In the example above, we find that only around 20 % of expected impact related to community infrastructure needs has been achieved, while 80 % on the impact is still to be achieved. Details of these provide us information on what can be celebrated and what still needs focused attention for

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 96: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

96future improvement through greater or modified effort. However in CGA-CHE's case this activity is used more as a way to provide some work opportunity as relief, after diasasters. In this sense the objective has been met.Code Number: PE- 05 Time required: 20-25 minutes

Name of the exercise: Macro-profile of expenditure analysis in the area of work.

Materials required: Flip charts, marker pens, seeds, masking tape.

Preparation: Read the Briefing notes on the TST, and after the focus group of Program staff responsible for this impact has been briefed about what is expected of them, proceed with the exercise.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 97: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

97Example:

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 98: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

98

6/15/2007 Ravi Jayakaran / Participatory Evaluations

6

Other Other Expenses Expenses (ceremonies, (ceremonies, gasoline etc)gasoline etc)

Health Health carecare

SchoolSchoolFoodFood

Macro picture – CHE Project area (76 villages)

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 99: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

99

Steps involved: Ask the Focus group to group the seeds together into various groups, depicting the various things that an avarage family in the community has by way of expenditire. After there is consensus on the ratios and components, a diagram is made as shown above.

Analysis of the information: In the example above, we find that around 40% of the community's expenditure is on food, followed by 30% on health care and 20% on education. Other expenditure related to transport, ceremoies etcetra account for another 10%. As a thumb rule, a community that has a third of its expence on food is considered to be a poor community. This community also spends a lot on health

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 100: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

100care, hence anything done to reduce the cost of health care will also have a major impact on the community.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 101: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

101

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 102: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

102

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 103: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

103

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 104: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

104

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 105: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

10546. Briefing notes on the TST (Ten Seed Technique) developed by Dr. Ravi Jayakaran (QPI-GMS®: (Extract from The ‘Participatory Poverty Alleviation & Development ‘ book written By Dr. Ravi Jayakaran and published by World Vision International China, April 2003 & Mekong Institute, Khon Kaen University, Thailand: CD ROM 2005.

INTRODUCTION: The 10 seed technique is a modified PLA tool and was introduced after a lot of modification and experimentation as a tool that can be used to carry out the PLA-Participatory Learning and Action exercises. It is useful in gathering qualitative information on various issues, especially related to the perceptions of the

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 106: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

106community and the way people see themselves in relation to others. The technique is very flexible and therefore versatile enabling its use in combination with other techniques and also for collecting a wide range of information. The 10 seed technique enables probing deeply into different dimensions of an issue, for carrying out what is referred to as “opening up” the information. This essentially involves going deeper into an issue after starting at the absolute basic level. For example we find out about the health status of a community and then go deeper into it to find out reasons for difference, link it with their wealth status and go further into exploring the type of health care each group is able to access. This ‘opening up’ process can keep continuing as we find linkages to education level and attitudes.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 107: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

107

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 108: Manual de evaluación participativa

The Left and Right Brain…

•Input device (H/S)

•GIGO

•Trained

•Schooled

•Formatted

•Filing: Systematic

•Limited by the ‘schooling it gets’

•Input device (H/S/T/T/S)

•Visual

•Perceptive

•Analytical

•Versatile

•Filing: ”creative”

•Immense capability

•Access to sub-conscious mind

LEFT RIGHT

Ravi Jayakaran/QPI-GMS/ DFID-UNDP-MoEYS training/Feb 2006

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

108

Most of the traditional PLA exercises can be done using the 10 seed technique; hence this will be explained in considerable detail. Detailed information can be obtained from the web sites mentioned at the end of this chapter.

PHILOSOPHY BEHIND THE TECHNIQUE: This technique relies on using the Right brain function so that the full potential of the brain for perceptive analysis is utilized. The right brain is initiated into action by visuals such as pictures, and three-dimensional items. When we use seeds to depict aspects of information, the visual created by the seeds (strong contrast of colors

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 109: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

109between the seeds and background), helps the Right brain play a dominant role in the analysis. In the beginning when this technique was developed, it was designed to enable illiterate villagers to participate in the discussions related to analyzing their situation. The idea was to remove differences due to literacy and enable those without literacy to be able to participate equally with those who were literate. However when the technique was used in the field, it was discovered that there were additional benefits from being able to activate the right brain. The right brain is the more powerful part of the brain, because it is the creative side, more perceptive, more analytical and also has access to the information both in the left-brain and the subconscious mind. Thus today the technique is even used with those who are

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 110: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

110literate! For best results, facilitation must aim at getting the group to ‘move the seeds first’ then describe the section…rather than preparing a list first and then trying to slot the seeds in!!

POTENTIAL USES OF THE TECHNIQUE: As has been already mentioned, the 10 seed technique can be used for a variety of exercises for information collection. These are for example: Profile of street children, Vaccination levels, Patterns of distribution among a population (health care, HIV/AIDS, birth control practices, etc.), disease incidence, agriculture patterns, animal husbandry practices, sanitation practices, housing needs, MED profile in an

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 111: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

111area, water resources, CEDC, rapid damage assessment in Disasters, education levels and, many other issues that project staff need to collect information about. Besides these, several of the routine PLA exercises can also be carried out by this technique for example: Trends analysis, Seasonality diagram, Livelihood analysis, Expenditure analysis, problem analysis, and etcetera. Some of the new exercises which are used for Wholistic World View Analysis (WWVA), and Capacity–Vulnerability (C/V) analysis, Rapid Food Security status assessment (RFSA), Gender desegregation, HIV/AIDS macro zoom, and District/County level planning also use the ten seed technique. These additional exercises have been designed because of specific needs in program management and other community needs.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 112: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

112Given below is a brief description of how the ten seed technique can be used for each of these exercises:

Trends analysis: The group is asked to think of all the things that have changed in their community since the past. They are then asked to list the things that have changed. In front of each issue that has changed they are asked to prepare two columns, representing the situation – “then” and “now’. For each area of Change, they are asked to use 10 seeds and distribute them between the ‘then’ and ‘now’ columns. The trend of change then becomes obvious and allow for a lot of discussions.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 113: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

113

Seasonality Diagram: The group is asked to carry out a seasonality diagram as in the standard practice, but with the change that for each of the seasonal events such as rainfall, agriculture, disease incidence, festivals, Labor opportunity etcetera, they are asked to use only ten seeds for distribution. This enables us to identify the occurrences according to percentage intensity at different times of the year.

Livelihood analysis: The group is asked to imagine the 10 seeds represent the entire income of the whole village from all sources, throughout the year. (To make it easier they are asked to imagine it all to be converted into money as some of it is

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 114: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

114generated in cash and some in materials). They are then asked to divide the seeds into groups representing the sources of the income. Use of the ten seeds here enables us to find out what the main livelihood sources are.

Expenditure analysis: The group is asked to imagine that the 10 seeds represent the total expenditure of the village for the whole year. They are then asked to group the seeds into clusters to show what those various expenditure heads are on a yearly basis. Again this allows us to determine the percentage of expenditure on various items such as food, clothes, and medical treatment, etcetera.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 115: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

115Problem analysis: The group is asked to imagine that the 10 seeds represent all the problems that are faced by the community as a whole. They are then asked to group the seeds to represent what these problems are.

Disease incidence: The group is asked to imagine that the ten seeds represent all the diseases that occur in the village throughout the year. Here too the numbers of seeds in each group show us the percentage of a particular disease in a year.

Wholistic World View Analysis (WWVA): This exercise is done by combining the Livelihood analysis and the problem analysis information of the village with the

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 116: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

116information on uncertainties in the village. This is an exercise that involves the whole village and is usually done after carrying out a debriefing to the whole village of information gathered in their village.

Rapid Food Security status assessment (RFSA): This technique is carried out using the ten seeds and the group is asked to classify the village families into different groups according to their Food Security Status and according to the periods which they have struggles generating their livelihood.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 117: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

117

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 118: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

118Gender desegregation: This exercise is carried out to identify issues that are gender related and to determine the extent to which there is a link of an issue or a responsibility with Gender. To find this out, the issue is first determined. Let us take for example the issue is the decision regarding Family planning. We then ask the group what extent of the decision on family planning is determined by the men and by the women. They are asked to divide the 10 seeds accordingly.

HIV/AIDS macro zoom: This is a method used to determine rapidly issues related to HIV/AIDS risk in a community. It is particularly useful for determining the strategy for HIV/AIDS prevention in a city. The group to work with is composed of around 25

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 119: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

119to 30 people drawn from different walks of life who are familiar with the city. These people are usually from the media, the health care services, the entertainment business, the law enforcement department, and students from the University, the transportation services and the NGO sector that has been working with HIV/AIDS prevention in the city. The series of exercises that are part of the Macro zoom look at the HIV/AIDS risk frame for the city and in the course of a day are able to make a rapid study of the potential strategies to be followed. The details of the exercises and examples are currently being documented by the author of this book, and should be available in due course of time.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 120: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

120

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 121: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

121

ADVANTAGE OF THE TECHNIQUE: The technique is very simple to understand and learn and equally easy to practice. When used with village groups also, they very easily understand it. It has been tested in several countries and has worked equally effectively in all of them. This technique has also been tested and found successful with people in different age groups varying from very young children to old people, with gender segregated groups and mixed groups and with people of no literacy and those with Doctorates!

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 122: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

122The other advantages are that seeds are easily available everywhere, very non-threatening, and easy to move and move again. Once the moving around of seeds stops with unanimous agreement the information can be written on a sheet of paper. The technique is a very visual one, and because of this allows for the literate and illiterate to participate as equal partners and contribute meaningfully to the discussion. The specific number of seeds enables the group to make reasonable comparisons. It is also possible because of this to determine approximate percentages. The resultant visuals are easy to explain, understand and discuss around.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 123: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

123How the technique works: After initial rapport building with the group and explaining to them that the purpose of the exercises is to understand and learn from them about their perspective; we proceed with the technique. The group is given the ten seeds and asked to consider them to represent the entire population under study. They are then asked to move the seeds around into groups representing the aspect being analyzed. Once the groups of seeds have been formed, the participants are asked to describe them and give details on reasons for classifying them the way they have done. Further details are then sought on indicators that determined the segregation. Each group of seeds now has a very

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 124: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

124Distinct identity accorded by the participants, and discussions can now proceed around the “visual” created. Discussions around the “visual” soon become very intense and animated. After finalization the information is transferred on paper for sharing with the larger group. The following slide shows an example of this: After this is done, we proceed further to ask more details, looking for

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 125: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

125example at the types of birth control measures used. The group is asked to look at the ‘visual’ and pointing to the 8 seeds representing the population that regularly practices birth control measures, we ask for them to divide these further in terms of the types of measures practiced. This can be done in two ways, either just asking them to divide the eight seeds, or by again taking ten seeds and asking them to consider these seeds to represent those who practice family planning measures regularly.The result of doing this can be seen in the following slide, where the group divided the seeds further into 5 groups showing the pattern of distribution. Thus, by looking at this it is possible to identifying what percentage of the overall population

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 126: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

126approximately uses a particular type of birth control measure. For example in the case above we found that the community was under the perception that they were quite well protected against HIV/AIDS, because a fairly high percentage of them were using birth control measures. They had, in their thinking, equated ‘protection’ against pregnancy as being protection against HIV/AIDS. Discussions around the visual then could proceed to understand why this was so.

Depending on the purpose of the exercise we can proceed further. As in the case mentioned above the purpose was to find out the condom usage prevalence rate, because another exercise in the same community had shown a fairly high level of

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 127: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

127promiscuous behavior among them. The exercise thus enabled us to see that only 30 % of the birth control measures in use were condoms, which also did not provide them 100% protection against the spread of HIV/AIDS.

Thus the exercise can also be the basis of discussions for modification of behavior when the community ‘discovers’ how much at risk it is. These discussions can also lead into understanding appropriate interventions in the community for modified or changed behavior. Other exercises can be linked to this to find out how the

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 128: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

128community gets its information on family planning and thus we can identify the best and most effective strategy for intervening in the community.

GROUP SIZE & PARTICIPATION: The ideal size of participants group is around 8-10 persons. Some would swear that the ideal size is 7-9 persons, but there is need to be flexible about this because these optimum sizes may not always be possible in the community, with the group sometimes being smaller or larger. In larger groups too the actual number of active participants may only be 8-10.If the number of active participants increases beyond that, and then it might be important to split the group further. Giving everyone an equal chance to share views can enable active

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 129: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

129participation. Sometimes it may be necessary to “filter out” the over dominating ones in the group, so that others can participate. Getting someone in the group to ‘diplomatically’ take the person away from the group for a KII-Key Informant interview does this. While this has the benefit that the group left behind becomes active in participation, good information can also be obtained from the person filtered out. Prime candidates for this type of filtering are usually Schoolteachers, Village chiefs, Businessmen, moneylenders and “educated” people in illiterate communities.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 130: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

130

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 131: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

131FACILITATING THE EXERCISES: Before going to the field to carry out the exercises, it is good to have the group prepare itself to work together. Besides establishing a code of conduct to respect and value one another, group members must also decide roles among themselves beforehand to determine who will be the ‘ Interviewer’, ‘the recorder’ and if the situation calls for it- the ‘Filter’!! As with other PRA/PLA exercises, facilitation involves ensuring that there is no dominance, and that everyone gets an equal opportunity to participate and that there is a balance between being open to new and divergent views as well as being focused enough to lead the discussions in the direction of the information being sought. Facilitation also involves determining when a dominant person has become the ‘defacto-

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 132: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

132spokesperson’ of the group and needs to be diplomatically ‘filtered’ out!! The facilitator (Interviewer) should also keep a balance of time required for the exercise so that it is neither hurried nor drags on too long!

KEY SUCCESS FACTORS: The correct attitude, a balanced group, adequate time and the ability to keep the group working with a right brain orientation are some of the Key success factors in getting good quality information. It is almost mandatory that the ‘outsiders’ carrying out the 10 seed technique exercises in the community have the correct attitude. This attitude is one of listening with a positive attitude. This essentially means being open to new perspectives and views without pre-

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 133: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

133assumptions which are merely seeking for an ‘affirmation’ from the community. A balanced group of outsiders with varied experience and one that does not those among it who are overly dominant, nor seek to dominate the community, is also important. Such a group will be able to enable the community to participate well and spontaneously. Adequate time should be available so that there is no need to rush. The exercise can then be coordinated and facilitated well, and focus on good discussions with openness to new views and perspectives.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 134: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

134It is also very important to keep the group working with a right brain orientation. This essentially means that at the beginning stages the seeds should be the main things on the paper with only symbols or diagrams drawn to mark the issues identified.

ASKING GOOD QUESTIONS: After establishing good rapport with the community group the facilitators of the exercise should explain the purpose of the exercise and the subject that they propose to explore with them. A foundational principle to remember in asking questions is to remember to ask questions to learn and understand, not to ‘affirm’ pre-assumptions. Questions asked should therefore be ‘Open ended’, and the 5W+1H (Who? What? Where? When? Why? + How?)

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 135: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

135Principle is a good one to adapt. The general principle is to start with the simplified foundational information first and then get deeper and deeper into the issue. Going deeper into issues to ‘Open up’ the information is a little more complex but comes easily with practice. Here too the best results are obtained by continuing to be as “visual” as possible and getting participants to move the seeds first and then describe the categories.

INTERVIEWING THE ‘VISUAL’ Once the ‘visual’ of the seeds placed in the different groups has been created it is reviewed with the participants to make sure everyone understands the placement of the seeds. After this various aspects of this are

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 136: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

136discussed…. this is done by “interviewing” the visual. This is a very non-threatening method, as we don’t need to make embarrassing eye contact with the participants especially when collecting sensitive information. When they feel comfortable and they are ready for it, they do however make eye contact, but this is on the basis of an established relationship. At the end of the session a participant from the community is invited to give a summary of the observations and findings. The document can be copied and the original left behind for use by the village in future.

PRECAUTIONS AND CARE: There are some precautions and care that one must take while using the 10 seed technique. The first and most important one in this is

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 137: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

137that the facilitator must have the correct attitude. This is Mandatory. Second, the purpose of the exercise must be explained to the community clearly right at the beginning.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 138: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

138

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 139: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

139Since the technique uses right brain function it has the potential to bring Hopes & Dreams to the surface and therefore something must be done to address them after the exercise. Similarly when seeking sensitive information, it may throw light on an exploitatory situation in the village that must be dealt with urgently. This too must be dealt with soon, or it will result in frustration and the oppression may continue unchecked. It is also essential to ensure equal participation and opportunity to speak for all participants without dominators ‘taking over complete’ control. Chapter six outlines ways in which to deal with such situations. Taking these precautions will enable the exercise to be done well and successfully.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 140: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

140POTENTIAL PROBLEMS: Dominant people in the community could be one potential big problem that will hinder good participation from the community. To prevent this, “Filter” dominators for KII (Key informant Interview) so that others get a chance to participate equally. As mentioned earlier, prime candidates for filtering are usually: Village leaders, moneylenders, Landlords, old school teachers, or other authorities. This, however, must be done sensitively so that the ‘filtered’ person still feels important. Besides, this is often a blessing in disguise, because the filtered person does in fact give some good information.Other problems can be avoided by being sensitive to the presence of people with vested interests in the group and ensuring that information is not biased in favor of their interests. Gender biased information

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 141: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

141should be countered by trying to get some gender balance. If the context is one in which women will hesitate to speak in the presence of men, and then divide the participants into 2 groups. Choose the timings for the exercise carefully so that it doesn’t disrupt the normal life of the village badly.

GOOD FACILITATION: This essentially involves being able to overcome some of the potential problems that one might face as mentioned earlier. The 10 seed technique is very easy to learn, but the correct attitude is hard to develop (hence we need to work on this if we want to be good facilitators). Developing openness to new perspectives and ideas and views is essential. One has to seek to listen and learn

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 142: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

142(not seek to confirm pre-determined ideas). Success is guaranteed when one develops a ‘listening attitude”. A good facilitator is sensitive to the local context and culture and ensures equal participation and counters dominance by individuals trying to become spokespersons for the group. If handled correctly, the exercises will generate animated discussions among the participants.

Let facilitation become a way of life for you…. not only at work, but also in your personal life!!

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 143: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

143‘OPENING UP’ THE INFORMATION:

This is done for issues that have ramifications with several factors for example the Food Security Status of a village in relation to migration patterns. The Food security status of the village is understood and then for each food security status level further details are found out for example level of children’s education, Health status, migration patterns, use of different programs etcetera. Carrying out this exercise results in the creation of a complex and complicated diagram. This diagram shows the ramifications and the underlying issues involved that cause the particular situation. Since the complex diagram emerged from the simple it is

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 144: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

144easy to understand, describe and discuss for all participants, even the villagers with low or no literacy.USING THE INFORMATION: Information generated by using the 10 seed technique must be used as soon as possible in planning and designing. However, oftentimes what really happens is that those who collect it are unable to analyze it and thus don’t know how to use it. When using technique for analyzing specific issues such as say the level of vaccination, the resulting information can be used to generate a Plan of Action for getting the unvaccinated children vaccinated. The participants of the exercise can be asked how to ensure that this happens for each level identified.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 145: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

145When the entire series of exercises is conducted, it must result in the formulation of a Village Resource Development Plan (VRDP) .The Problem analysis and Livelihood analysis exercise of the village in combination with the uncertainty profile of the village can be used to prepare for the Wholistic World View Analysis (WWVA), which in turn can help is generating a C/V (Capacities & Vulnerabilities) analysis of the village for Development planning. Sometimes the exercise is done to identify the causes of a specific problem. Thus the information generated can immediately be used by addressing the identified causes. On other occasions the exercise is done to find out why a particular program is not succeeding…. the approach here would then be to make modifications as suggested by the community.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 146: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

146

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 147: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

147

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 148: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

148The progress of the program against planned objectives can be done in the form of an evaluation .For example for a particular objective if the planned impact was ‘x’ and we represent this with ten seeds. We then ask the community how many seeds would represent the actual impact, and let us say they put 7 seeds to show this; we then ask them to list these. Similarly, we can then proceed to ask them what modification and emphasis (3 seeds) needs to be given in the project for that component in future to bring it up to speed. This could even be done at each output level for the project.

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 149: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

149CONCLUSIONS: The range of uses that the 10 seed technique can be put to have been clearly detailed in this book and the one specifically related to the TST. Their uses range from participatory information gathering to situation analysis, planning and evaluation. Extended applications have also been explained for example on how it can be used to deal with specific issues such as HIV/AIDS risk, Food Security Status, understanding the needs of children etcetera.

Web site references:

http://www.fao.org/Participation/ft_show.jsp?ID=1981

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 150: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

150http://www.childsurvival.com/connections/start.cfm#spring2002http://www.fupin.org.cn/ngohy/ngo-z/ngo-z-show-16.htmhttp://www.mrcmekong.org/info_resources/infores002b004.htmhttp://www.hivmekong.net/Downloads/Toolkits/Toolkit_Final.pdf

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 151: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

151

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 152: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

152

Glossary of some terms used in this book:

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 153: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

153

Additional reading on the subject:

No Title Author

01 Rural Development – Putting the last first(Longman, Harlow)

Dr Robert Chambers1983

02 Participatory Workshops(Earth scan)

Dr Robert Chambers2002

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Filtering The process by which a dominant person is diplomatically removed from a group

FGD Focused Group Discussion

HIV/AIDS Human Immuno deficiency Virus/ Acquired Immuno Deficiency Syndrome

KII Key Informant Interview: Where the source of information is a single key informant

Optional ignorance

Choosing to ignore certain aspects of information that are not relevant to analysis or planning

Optimal ignorance

The maximum that one can safely ignore in investigating

OSPE Out of school Peer Educators

OSP Out of School Peers

PRA Participatory Rural Appraisal

PLA Participatory Learning and Action

PPE Primary Peer Educators

PP Primary Peers

PLWHA People Living With HIV/AIDS

PPA&E Participatory Process Assessment & Evaluation: a technique by which we analyze where the development process or intervention is at in the ‘continuum’. This ‘in-course’ assessment enables us to see how the project is progressing.

SSI Semi Structured Interview: This type of interview involves leading the interview in a particular direction with questions that first start off on general issues and then slowly focus on specific issues within the general frame.

SPE Secondary Peer Educators

SP Secondary Peers

TST Ten Seed Technique

Triangulation The process by which other perspectives are also sought to ensure that the information collected is not biased or one sided.

Page 154: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

154

03 Whose reality counts? Putting the first last(Intermediate Technology publication)

Dr Robert Chambers1997

04 PRA/PLA Notes series, (especially RRA notes 13), London, 1991

IIED: International Institute of Environment & Development

05 PLA User guide & Manual(World Vision India publication)

Dr Ravi Jayakaran1996

06 PLA – a five minute guide(WVI / AusAid HIV/Aids toolkit)

Dr Ravi Jayakaran1996, 2001

07 Use of PLA for collecting sensitive information(AIDSCAP – India)

Dr Ravi Jayakaran1998

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 155: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

155

08 TST- The Ten Seed Technique(WVI China/ WVA downloadable) English/Chinese

Dr Ravi Jayakaran2002

09 Participatory Poverty Alleviation & Development(WVI China/ Mekong Institute, KKU, Thailand)English /Chinese

Dr Ravi Jayakaran2003/2005 CD ROM

10 Facilitating Small Groups(WVI China/ Mekong Institute, KKU, Thailand)English / Chinese

Dr Ravi Jayakaran2003/2005 CD ROM

11 A Trainer’s guide to Participatory Learning and action(IIED series)

Dr Jules Pretty Et all1995

12 Stepping stones: A training package on HIV/AIDS, Dr Alice Welbourne

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 156: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

156gender issues, communication and learning skills(Action Aid, London)

1996

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 157: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

157

A short note on writing case studies:

Case studies are the best way of getting deeper into an issue to understand the situation as well as the circumstances. This is particularly useful in determining the impact that has taken place in the life of an individual. As with any good interview, use the 5W and 1H principle to determine the questions that you will ask to find out

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 158: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

158as much about the person and his/her situation and circumstances. Since there is likely to be a lot of sensitive information that will be generated from this type of case study, take the following precautions:

Don’t use the person’s real name if issue is sensitive (change it for a fictitious name)

When taking photographs ensure that the person’s identity cannot be determined from looking at it

In the case study try to determine specific instances and decisions that resulted in action on the part of the individual, to understand the mental profile and mental process that precipitated the action

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 159: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

159 The case study also throws up some excellent perspectives on the ‘realities

on the ground’

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 160: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

160

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 161: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

161

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 162: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

162

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 163: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

163

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 164: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

164

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 165: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

165

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual

Page 166: Manual de evaluación participativa

Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International

166

Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual