manual de evaluación participativa
TRANSCRIPT
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
1
© Dr. Ravi I. Jayakaran, QPI-GMS®, June 2007 Prepared for the CHE program team working in 76 villages in Cambodia as part of Cambodia Global Action.Made available through special funding support from Samaritan's Purse International
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
2
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
3
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
4
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
5
Photographs in this manual are by author unless otherwise specified.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
6
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
7
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
8
This manual is dedicated to the poor and marginalized in Cambodia, and to the task of equipping CGA staff to become good Participatory Evaluators who
will continue to seek to improve the performance of their programs and be committed to Continueous Quality Improvement (CQI).
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
9
A user Guide and Manuall
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
10
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Photo: Joanna Bryden
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
11
Chief Consultant: Strategy & Development Programming
Quality Professional Interfaces-GMS® Cambodia
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
12
Index:
Chapter Subject Page number
1. Introduction
2. Cleaning up the design Matrix
3. Background and Principles of Participation
4. The Uniqueness of Qualitative information
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
135. Types of groups to be interviewed and documentation
6. Using the Manual
7. Scheduling & Evaluation design
8. General information at village level
9. Purpose of Evaluating Programs
10. Stage when Evaluation should be done
11. Strategy for Evaluation (How)
12. Preparing for an evaluation:
13. Stakeholder analysis
14. Difference between Monitoring & Evaluation
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
1415. Annual reviews and Ministry Reviews
16. CQI and Strategic Planning
17. Determining what to do
18. Planning ways to do it
19. Designing a tailor-made Evaluation
20. Various Participatory Exercises
21. o Country profile
22. o Impact-1: Community health improved
23. o Impact-2: Family Food security status improved
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
1524. o Impact-3: Increased capacity to manage their own lives
25. o Impact-4: Community infrastructure provided
26. o Expenditure analysis – at Macro level
27. o Uncertainity analysis – at Macro level
28. o Problem analaysis – at Macro level
29. o Livelihood analysis – at Macro level
30. o Wholistic Worldview Analysis (WWVA)
31. o Community Attitude to CHE program
32. o Community Attitude to Christ
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
1633. o Community Attitude to the Change agent
Index (contd…)
Chapter Subject Page number
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
1734. o Grouping of villages according to their performance
35. o Key stakeholders in the development process
36. o Stakehoder analysis (in a successful village)
37. o Stakehoder analysis (in a partly successful village)
38. o Stakehoder analysis (in a village with low results)
39. o Uncertainity Analysis at village level
40. o Problem Analysis at village level
41. o Livelihood Analysis at village level
42. o Expenditure analysis at village level
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
1843. o Community level program inventory
44. The Default Priority Profile - DPP
45. The Value Change Index
46. EEIRS (Efficiency, Effecitveness,Impact,Relevance, Surstainability) test for the overall performance of the organization
47. Acronyms used in this manual
48. Conclusion
49. Notes on the TST
50. Additional reading
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
19
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
20
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
211. Introduction:
Samaritan’s Purse International Relief (SP) is Christian Non-Governmental Organization that has been providing relief to people in need around the world since 1970. For the past 23 years, SP has helped meet the needs of victims of war, poverty, natural disasters, disease and famine. SP responds quickly, appropriately and has been active in over 100 countries worldwide. Although largely donor funded, SPIR has successfully partnered with donors such as USAID, CIDA, DFID, UNICEF, WFP and UNHCR.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
22For almost 10 years, SP has been helping to provide financial and human resources to a variety of local and international NGOs operating in Cambodia. Most recently, SP has ventured into a partnership with Cambodia Global Action (CGA) to grow their CHE programming with animal husbandry and village credit programmes. The Seeds of Hope programme offers seed resources for animal banks, impact monitoring and CHE trainings to over 80 CHE committees throughout Cambodia. It is SP’s long term desire to open this programme to other CHE-based projects being managed by other LNGOs in Cambodia. This relationship with CGA is considered a pilot of more expansive programmes for others working with the CHE methodology.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
23CGA is a local Christian NGO with CHE programming for over 5 years in 76 villages across 9 program units with 16 sub project units at a commune level.
1) Kompong Speu 1. Kandal Dom Commune, Chbaa Mon District2. Mohasaing Commune, Phnom Srouch District
2) Kandal (Khien Svay) 3. Dey Eth Commune, Khien Svay District4. Bantey Dek Commune, Khien Svay District
3) Kandal (Saang) 5. Svay Rolum Commune, Saan District6. Kos Anlong Chin Commune, Saang District
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
244) Kampong Cham 7. Teuk Chroow Commune, Dambae District
8. Chong Cheach Commune, Dambae District9. Nean Toet Commune, Dambae District
5) Phnom Penh (Dangko) 10. Trapean Kransang Commune, Dangkor6) Kampot 11. Andong Khmae Commune, Kampong Bay
12. Meak Prang Commune, Kampot District7) Siem Reap (Pouk) 13. Keo Por Commune, Pouk District8) Takeo (Bati) 14. Trapeang Krasang Commune, Bati District9) Stung Treng 15. Samahki Commune, Stung Treng District
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
25
This Manual has been Tailor-made for the CHE program of CGA but has principles that can be adopted in other programs too.
2.Cleaning up the deign Matrix (Making the Log frame Evaluation friendly):
Before carrying out an Evaluation, one must ensure that the design Matirix (PDM-Project Design Matrix) that forms part of the Log frame for the project's programs is properly streamlined so that activities result in Out-puts of significance, and that these in turn end up together having an Impact. A series of 3 or 4 Impacts then lead
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
26in the direction of enabling the Project or Program achieving its Objectives or Goals. Evaluations are carried out at the Impact level, hence, if the Design Matrix isn't properly integrated, it will be difficult to Evaluate or attribute success to any particular program. In reviewing the design of the CHE program of CGA, we found that there was some misalignment of the activities, Outputs and impacts. Time was therefore taken to re-align these so that they fitted more correctly into a logical sequence. The details of the revised Matrix are seen below:
Impact I: Improved Community Health AchievementOutput I.1: Improved Primary Healthcare
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
27
Activity I.1.1: Community Health Screening
Output I.2: Improved Water/Sanitation
Activity I.2.1: LatrinesActivity I.2.2: WellsActivity I.2.3: Soap DistributionActivity I.2.4: Water Filters
Output I.3: Improved Nutrition
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
28
Activity I.3.1: Home GardenActivity I.3.2: Rice BankActivity I.3.3: Rice Storage
Output I.4: Improved Environment
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
29
Activity I.4.1: Organic Pig PenActivity I.4.2: Earth Worm RaisingActivity I.4.3: Garbage ManagementActivity I.4.4: Compost PitActivity I.4.5: Fruit Tree NurseriesActivity I.4.6: Bio Gas
Activity I.4.7: Natural Pesticide
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
30The same principle was applied to all the Impact areas as follow:
Impact II: Family Food Security Increased Achievement Output II.1: Micro-Enterprises Increased
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
31Activity II.1.1: CreditActivity II.1.2: Rice Straw MushroomActivity II.1.3: Soybean SeedActivity II.1.4: Self Help GroupActivity II.1.5: Food ProcessingActivity II.1.6: Fish Ponds
Output II.2: Animal Raising IncreasedActivity II.2.1: Chicken RaisingActivity II.2.2: Cow BankActivity II.2.3: Goat BankActivity II.2.4: Pig Bank
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
32
Impact III: Increased Capacity to Manage Their Lives Achievement Output III.1: Improved Spiritual Lives
Activity III.1.1: Video EventsActivity III.1.2: Education
Output III.2: Improved Basic EducationActivity III.2.1: Village Livestock Agent Projects Activity III.2.2 Literacy ClassActivity III.2.3: TrainingActivity III.2.4: Family Competition
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
33 Output III.3: Improved Community Structure
Activity III.3.1: VDCs, PMCs, CHEs and CHE ModelImpact IV: Community Infrastructure and Relief Provided Achievements Output IV.1: Community Infrastructure Created
Activity IV.1.1: Road Repairing (Food for Work)Activity IV.1.2: Water Reservoir (Food for Work)Activity IV.1.3: School Construction with Community
ParticipationActivity IV.1.4: Sewage System with Community
Participation
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
34
Once this sorting out was done, it became relatively easy to decide what the framework for the Participatory Evaluation should be.
3. Back ground and Principles of Participation:
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
35
The TST (which has been described in detail in the briefing notes in Chapter 46) is used a great deal now in the Greater Mekong Sub-region for Participatory development programs. One of the biggest advantages with this technique is that it allows for discussion of information that is otherwise sensitive to handle. Besides this, while using the seeds and grouping them to depict information, there is the creation of a visual that tremendously enhances the quality of discussions. Also because of their ‘non-threatening nature’ the seeds allow group members to feel free to express themselves openly and frankly. It is important while facilitating the group discussions to ensure that all members in the group get a fairly equal opportunity to share their views and that they are not dominated over by any group members and
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
36coerced into agreeing to a ‘uni-polar’ view. This therefore means that the group should be of an optimum size of 7-9 members (discussed again later) for best results.
Facilitation is an art. In fact it is a way of life! Care must be taken to ensure that the core evaluation team members get ample input to develop their capacities to become good facilitators. This applies also to those who will be added to the core team in later months. All of them must go through the basic training and go through the pre requisite orientation to ensure that they do not become ‘manipulators’ of opinions, but rather become good facilitators who are able to bring out from the
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
37group they are facilitating, their true and real opinions. The purpose of Participatory Evaluation is to get an honest picture of what we are doing and the type of impact we are having…not to fool ourselves into believing that everything is perfect!!
While facilitating, ensure that there is adequate time available for the exercise. This involves planning well in advance and ensuring that all members of the group are comfortable. It is often a good idea to have some fruit available for community participants especially as the day moves on close to lunch time or toward the evening. Before starting the facilitator ensures that at least one person has been given the task of documentation of all the discussions. Working in teams is important
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
38for this reason. After the group has been comfortably seated, introduce the purpose of the exercise, the methodology and then proceed. All other prerequisites by way of materials required, and preparation are mentioned in the notes for each exercise.
4. The uniqueness of Qualitative information:
Qualitative in formation has a different mandate from Quantitative information. This must always be remembered by the Evaluater. While preparing the report one must capture the uniqueness of the context that will emerge when facilitation is done well.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
39Each site must have the same number of exercises conducted for uniformity. Unlike quantitative information, the outcome of the exercises will be a narrative document with the visuals of each exercise in it. Comparing between sites is therefore is not so much a matter of something being ‘better than’ or ‘worse than’ but a matter of looking for linkages and learning from the composite information.
Qualitative information allows for obtaining width as well as depth to the information and this is what gives it its richness. To begin with, the principle is to ‘cast the net wide’ and get as much width from the information as possible. When something is seen to be significant, it is good to go into depth and probe as much as possible for
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
40hints of reasons and causes. In some cases it may even be possible to carry out case studies of people who are seriously affected. This often throws light on reality in a most unusual way. For example, we may facilitate through awareness and capacity building, ‘sexual negotiation skills’, but the whole issue of condom availability in the village may be the biggest stumbling block. Again, even if condoms are available in the village, who they are available with may become an issue! Again in conservative society, it may be difficult for girls to ask questions of clarification on issues that are not immediately clear, and the findings may be to just go ahead and list some of the questions that ‘normally arise’ and answer them.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
415. Types of groups to be interviewed & Documentation:
The Evaluation process involves interviewing individuals, and small homogenous group such as a government department where only one person may function as a spokesperson or with a small group where everyone is free to participate. The technical term for the first type is ‘Key informant interview’ or KII. The second type is referred to as a ‘semi- structured interview’ or SSI, and the third one is referred to as a focused Group discussion. The exercises designed for this manual are all specifically designed for Focused Group Discussions (FGDs), but they can also be used for the SSI and the KII. The dynamics of group interaction allows for
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
42triangulation (correction and modification by others in the group) of the information.However in both KII and SSI there is no scope for Traingulation. The ideal size of a FGD is 7 to 9 members. Care should be taken to get a random representation along with reasonable gender balance and age balance (when relevant). Those selected must have a good idea about the topic so that they can all participate and contribute to the discussion. In the design of each exercise, it has been recommended (where relevant) to also discuss and share the findings of the FGD with a larger group to ensure that they also share the same views. When this is not the case, a note should be made in the document of the changes that were suggested. Also, ensure that the original flip chart is left behind with the members of
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
43the FGD or some responsible person at the site. This means that the documenter must copy the final visual also as part of her/his documentation.
Documentation: Each exercise has a code that begins with ‘PE’ followed by a number. During the Pilot assessment, orientation and training, the core team will be shown how to use this exercise code and also determine some guiding questions that would be useful to facilitate the FGD. While the facilitator is facilitating the group discussion, the documenter will note (as bullet points) all the important points that emerge. At the end of the discussion he/she will also copy the visual. At the end of the day, each team will look at the exercises that it has completed and read through
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
44the documentation, highlighting the important points. This also helps edit out points that have repeated themselves in the course of documentation. The highlighted points are synthesized into a summary that is about 1-2 paragraphs in length. To summarize, each exercise will have the following:
The Code with guiding questions A copy of the visual A transcript of the discussions with important points highlighted A summary of the important points.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
45The summaries (written in sequence) will constitute the site summary. The original copy of the visuals and code with transcript (as mentioned above) will be preserved in a box file at the main office for reference when required. As mentioned earlier, the original flip chart on which discussions were carried out and which has the original ‘visual’ must be left behind at the site with the group interviewed.
6. Using the Manual:
The following are the various parts of each exercise, as described in the succeeding chapters. Each chapter uses the same format. This has specially been written this
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
46way so that it is extremly user friendly and each exercise has all the details of what needs to be done. Every facilitator must study this clearly and understand the design.
7. Scheduling and Evaluation design:
The following exercises will require between 2-3 full days for one team to carry out in a village. The ideal team will consist of three members, whose roles will be interchangeable:
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
471. A team leader: to help with logistics, encourage group members to
participate, counter domination, and also prepare for the next exercise so that there isn’t any wastage of time in transitioning from one exercise to the other. The team leader will also be responsible for 'Filtering off'and dominant people in the group and for documenting case studies when they emerge.
2. A Facilitator: to facilitate the exercises as planned
3. A documenter: to document the details of the discussions
Alternately, if the PE team members work in teams of 5 and are located in the same place during the assessment, they can have one common team leader for two teams
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
48of two each. Each of the teams will have a facilitator and a documenter. The schedule for carrying out the exercises and the groups with which it should be carried out should be planned out in advance, before going to the village.
After completing the exercises the following is the Filing profile for individual exercise reports:
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
49
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
50
Shown above is the format used for the recording of the details of each exercise. These original documents should be stored in a box file at the CHE/CGA office in Phnom Penh. The code mentioned below in the summary as well as the individual
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
51exercise documentation enables the reference of the original papers when additional or detail information is sought.
Coding pattern:
PE-01
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation
Exercise number
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
528.General Information (at Village Level): At each site, obtain the following information that is relevant to the site and will help the team understand the details of what is happening in the process at the particular site. The following format shows how to fill the information and the last column also shows where the information can be obtained:
Details of information Source
Name of the Village staff
Date of starting the Program staff
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
53
Type of School ( Primary / Secondary) Village chief
Total land area of the village Village chief
Population density = Popn./ land area in KM2 Calculate
Infant mortality rate Calculate
Vaccination levels Use TST
Ratio of Doctors to population Calculate
Average rice production per hectare Village chief
Average rainfall in the area School Atlas of Cambodia (P.12)
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
54
Poverty rate (Neik Krao) RFSA
Poverty rate (Neik Tual Krao) RFSA
% of population with access to clean water TST
% of children in school TST
% of community migrating out every year TST
Visible display of acknowledgement of donor contribution to the Program: (Y/N)Note: Photo copy this template and get information for each village where Participatory Evaluation is being done.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
55
The general information on each village will enable the Evaluators to see what progress has been made in the village over the years and also enable the identification of areas which are still below standard.
9.Purpose of Evaluating Programs: If there is one thing that gets neglected in the project and Program cycle, it is the Evaluation of the impact of the program at periodic intervals. When a program is launched, it is done on the basis of some assumptions. These assumptions of imact are assumed to be the result of a group of outputs, which in turn are the result of a group of activities. When the program is actually put into place, some of these assumptions may not work out, or the impact
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
56may turn out to be different from what was intended. If the program is however evaluated every two or three years, the actual impact can be determined and this will help the management detrmine if the startegy employed was appropriate or not. Accordingly, corrections can be made in the program for bringing about improvement, dropping activities that are not relevant, or introducing new programs that may be required for corrective action. An evaluation is an excellent opportunity to look back at what has been achieved against the planned objective or goal of the program. It enables the Management of the program to see what progress has been achieved and celebrate it. The next section will talk about the time and stage when various evaluations are possible in greater detail.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
57
10.Stage when Evaluation should be done: There are different stages when an evaluation can be done. Smaller, and more focussed evaluations can be done at various stages of the Project or Program cycle as seen in the diagram.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
58
EVALUATIONS
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
59
For this, various aspects of the program cycle such as needs assessment, Planning, Implementation and monitoring system in place are evaluated. This type of Evaluation is manily done when there are implications of a new methodology or a program that requires wider implementation, and for which the initial 'pilot' program is to serve as a model. These evaluations are shorter in duration and may require people who are specialists on the program component that is being evaluated.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
60
PHASE OUT
3YRS
EOP-EProgram Life cycle and Evaluation
SEED PHASE 1ST 5 YEAR DESIGN
2YRS 5 YRS
QUALITATIVE(PLA)
QUANTITATIVE(30 CLUSTER HHS
SECONDARYDATA
E-1
1ST 5 YEAR DESIGN 2ND 5-YEAR DESIGN
5 YRS 5 YRS
E-2 E-4
E-1
E-3
The third evaluation helps plan the phase out design of the project.
Dr.Ravi Jayakaran: WV Cambodia
E-2 E-5
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
61
For larger, long term projects and programs, it is necessary to build in Evaluations at the end of each phase. Ideally, evaluations should be done at a 2-3 year interval, which means that the longer phases will get an additional mid-term evaluation also. Often, with projects and programs, the weakness is that at the time of writing the proposal, not provision was made for evaluations. They then try to look desperately for a budget when they require and evaluation and it often ends up as an unplanned, shallow and shoddy affair. Ensure that you budget for evaluations at regular intervals when you design your program. This will ensure that planning is also done with evaluations in mind, namely that program people will always be conscious that the
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
62program will be evaluated for impact. Evaluation reports when positive can open up the door for new and additional funding.
11.Strategy for evaluations: The Program design Matrix is the framework around which the evaluation is conducted. This therefore requires for the design to be well thought through and for activities, outputs and impacts to be integrated. In an earlier chapter we discussed that this needs to be 'cleaned up' if not done already before starting the evaluation. The principle then is to start with the Macro and then move down to the output level. As in the case of the CHE PDM, there are 4 main impact areas. The first step therefore is to see what impact has been achieved under each
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
63impact area, namely improving health, improving household food secuity, increasing capacity to manage their lives and creating new infrastructure. Changes in the lives of the community in these areas is explored and studied. Evaluations can be done based on quantitative data or qualitative data or a combination of both. For quantitative data to be of use, one must however have details in the form of a 'baseline' carried out at the start of the project. This manual details how to use qualitative techniques for carrying out an evaluation.
12.Preparing for an Evaluation:
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
64In preparing for an evaluation, one has to determine what aspects of the impact of the project are to be evaluated and by what means. For some of the aspects, there will be quantitative data available in terms of the numbers of deliverables that ahave already been generated by the program such as wells, or water filters, or latrines etcetra. This data should be documented for each of the activities and outputs under each impact area. These are available for review in the Appendix that accompanies this document. These hard numbers will show how progress has been made over the years and will especially be useful for the donor to be certain that their money was used wisely in producing the outputs that were planned. The evaluation doesn't stop here, but goes further to also measure how those outputs which were
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
65generated as part of the project activity impacted the lives of the community that was part of the target of the community. Staff teams are made and then they go along with external representatives to places other than where they normally work, to carry out the evaluation. Where possible, case studies are also developed of examples of both success and failure, so that they can be valuable lessons learned. After the evaluation is scheduled and starts, the external evaluator randomly visits some of the sites, to see how objectively the evaluators are doing their work. The exposure to other areas of work within the project also gives the staff a chance to see how their colleagues are working. There is cross fertilization of perspectives and they also
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
66draw new insights and lessons from the whole process. At the end of the evaluation, there is opportunity to debrief and share ideas and lessons learned.
13.Stakeholder analysis:
Another pre-requesisite for the evaluation is to carry out a 'stakeholder analysis'. This consists of finding out who the potential stakeholders are for the information that is likely to emerge from the evaluation.The stakeholder analysis is done using the TST and determining details with a focus group that consists of members representing the donors, the Board, the Management team and if possible one of the
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
67community leaders from the target area. The following page shos an example of how a stakeholder analysis looks. In the process of this Analysis we try to assess what each stakeholder's 'stake'is the outcome of the evaluation. We also find out what their principle interest is in the evaluation and this enables us to determine the framework for the evaluation and identify the issues that need to be studied.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
68
•For self assessment and for improving impact and program modification
Staff
• For improving further collaborationOther partners
•As an exercise to empower them and for greater accountability
Community benefiting from the program
•For ensuring there is good impact and for CQI (Continuous Quality improvement
Management team
•To ensure that the program is progressing well
Board of Directors
• To assess how well Fund was given
•To consider future fundingDonor (existing and potential new ones)
Remarks if anyWhat they are interested in
Key stakeholders
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
69
At the end of the evaluation, this stakeholder analysis enables us also to detrmine if we have found out all the information we intend to find out about.
14.The difference between Monitoring & Evaluation:
While the words Monitoring & Evaluation are often used together, there is a big difference that exists between the two of them. Monitoring is something that is ofeten done by the staff involved with the program activity and their supervisors. While the supervisor doesn't normally do all the monitoring, he or she should slowly develop a system of keeping track of progress in terms of completion of activities and what has
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
70been achieved in less measure than was intended. Monitoring is essentially a largely self'done and self assessed activity. Monitoring is also done at the output level, and this may be done at 6 monthly or annual intervals, as compared to the previous one in which activities are monitored at a monthly or quarterly interval. Evaluation, on the other hand is done at a 2-3 year interval when it is actually possible to assess what the impact of the program has been on the lives of the people. Some people also
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
71
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
72add an addition dimesion to their assessment by having an 'annual review'. This essentially consists of have a look at all that the program did over the past year and documenting the results of these.
15.Annual Reviews and Ministry reviews:
When there are several ongoing programs and activities continue over several years, it also makes sense to introduce a 'Ministry review' every 5 years or so, to study and recognize what the program is actually busy with, and if parts of it can be changed and reviewed. As with evaluations, It is always good to get an external
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
73person to facilitate the 'Ministry review', whicle the annual reviews can be done by existing staff with the rpecaution of getting them to review in places other than where they work on a daily basis. Both these exercises should be followed by reflection and correction to programs and activities, and ofcourse, a celebration of achievements.
16.CQI & Strategic Planning:
The annual reviews and the evaluations result in corrective changes for improvement of the focus of the programs and for greater efficiency and impact. These corrective changes facilitate what is known as a process of 'CQI' or Continous Quality Improvement. The Evaluations and the Ministry reviews on the other hand
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
74feed into the process of Strategic direction planning or Strategic planning and ofetn usher in major changes in the programs, enhancement of activities or changes in the program portfolio so that the organization achieves its goals to a greater extent.
17. Determining what to do:
It is important to determine what to do in advance and schedule this carefully. Good planning will enable the evaluation team to achieve much in its time of evaluating. When an organization experiences evaluations for the first time in its life, and there isn't an evaluation design available in advance, it might be useful to carry out a pilot
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
75assessment to determine what needs to be done. This method was adopted in the CGA-CHE program that had not had an evaluation in its several years of being in operation. Carrying out the pilot assessment helps in puuting together a 'tailormade'design for the evaluation process.
18.Planning what to do:
After the list of exercises are determined, the evaluation team meets together in preperation and all members are given necessary orientation and or training for carrying out the same before they go to the variuos sites for the evaluation.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
7619.Designing a tailor-made evaluation:
As described above, the evaluation of the CGA-CHE program was first subject to a pilot assessment, to determine what programs should be assessed, and then based on what was practically possible the evaluation framework and design were tailormade for the needs of the organization. Once established, this can now become the norm for future evaluations, with specific 'additional exercises' being added to this based on information that may be required to write future program proposals or to fulfill donor needs.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
7720. Various Participatory Evaluation exercises: Based on the pilot assessment, it was determined that some 25 Participatory evaluation exercises be designed for collecting the information that is required by the program and potential donors for future programming and funding. By carrying out these exercises, the program will be able to generate information on various aspects of its work to be able to determine impact and also identify areas which need to be discontinued or modified for better impact. Failure is not something to be ashamed of or worried about. To quote a famour innovater, "Errors are not mistakes, they are innovations gone wrong". Every failure is thus an opportunity for change, modication and improvement!
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
78
The following are the detailed descriptions of the exercises that can be done in the field for getting needed information. Each exercise has descriptions and an example from actual use in the field:
21.Country profile: This exercise consists of carrying out a macro profile of the country where the program is operating. In generating some of the basic demographic details of the country the Program team mebers learn where to get
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
79various bits of information. For Cambodia the main sources from where information can be obtained are: The Ministry of Planning, The National Institute of Statistics, The Ministry of Education, Youth and Sports (MoEYS) and from the Atlas of Cambodia and the school Atlas of Cambodia (available at Monument book stores).
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
80
6/15/2007 Ravi Jayakaran / Participatory Evaluations
1
1:3124Health facility (doctors: Population)
252Total number of High schools
670Total number of Secondary schools
1429Total number of Primary schools
8628Total number of schools
74Population density/ Square KM
35Poverty rate (extreme poor)
79%Literacy rate (15-44 age group)
1400mmAverage rainfall (annual)
1967/hRice produced (Kg /Hectare)
13694Total villages
185Number of Districts
24Number of Provinces
In CHE’s cover areaIn all of Cambodia
Details
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
81
Generating this information in the format shown above, and matching it with information from the full area where the CGA-CHE program operates, enables the team to see which areas it has been able to achieve a critical concentration presence for 'tipping the balance'in favour of change.
Besides doing this at a countryoffice level, the program should also fill details in at each field centre level. This information also comes in handy for lobbying for change and advocacy with the decision makers for changing the situation in a particular area.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
82
Code Number: PE- 01 Time required: 20-25 minutes
Name of the exercise: Impact-1: Impact of Improved community health.
Materials required: Flip charts, marker pens, seeds, masking tape.
Preparation: Read the Briefing notes on the TST, and after the focus group of Program staff responsible for this impact has been briefed about what is expected of them, proceed with the exercise.Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
83
Example:
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
84
6/15/2007 Ravi Jayakaran / Participatory Evaluations
2
• Some people still don’t have latrines
•Don’t wash hand regularly
•Some don’t have water filters
•Still lack clean water for all
•Children still dirty
•Some people still don’t have a rubbish pit
•Use latrines
•Know how to prevent disease
•Use water filters
•Reduced water Bourne disease
•Multiply knowledge on health
•Clean environment
•Reduced health expenses
•Increased vaccination
•Get ANC (Ante Natal care)
DetailsTSTDetailsTST
Yet to be achievedAchieved already
Macro picture in the 76 villages where CHE works
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
85
Steps involved: Ask the Focus group to group the seeds together into 2 groups, one depicting the impact that has already taken place and the other showing the impact that is yet to take place. After there is consensus on this, the group is asked to give details of actual and specific impact in terms of new acquisitions, innovations and changes under impact achieved and areas of shortcomings in the second group.
Analysis of the information: In the example above, we find that around 60 % of expected impact has been achieved, while 40 % on the impact is still to be achieved. Details of these provide us information on what can be celebrated and what still needs focused attention for future improvement.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
86
Code Number: PE- 02 Time required: 15-20 minutes
Name of the exercise: Impact-2: Impact of Improved Houshold foodsecurity status.
Materials required: Flip charts, marker pens, seeds, masking tape.
Preparation: Read the Briefing notes on the TST, and after the focus group of Program staff responsible for this impact has been briefed about what is expected of them, proceed with the exercise.
Example:
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
87
6/15/2007 Ravi Jayakaran / Participatory Evaluations
3
• Some people are still lazy to start of their own business
•Some people still continue to use the old techniques
•Some people don’t want to have their own home gardens
•Some people are able to establish their own business
•Started using the techniques
•Some people are able to grow vegetables for their families for the whole year
DetailsTSTDetailsTST
Yet to be achievedAchieved already
Macro picture in the 76 villages where CHE works
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
88
Steps involved: Ask the Focus group to group the seeds together into 2 groups, one depicting the impact that has already taken place and the other showing the impact that is yet to take place. After there is consensus on this, the group is asked to give details of actual and specific impact in terms of new acquisitions, innovations and changes under impact achieved and areas of shortcomings in the second group.
Analysis of the information: In the example above, we find that around 40 % of expected impact has been achieved, while 60 % on the impact is still to be achieved. Details of these provide us information on what can be celebrated and what still
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
89needs focused attention for future improvement. In Foodsecurity status improvement, it is obvious that a lot of improvement is required.Code Number: PE- 03 Time required: 20-25 minutes
Name of the exercise: Impact-3: Impact of Increased capacity to manage their own lives.
Materials required: Flip charts, marker pens, seeds, masking tape.
Preparation: Read the Briefing notes on the TST, and after the focus group of Program staff responsible for this impact has been briefed about what is expected of them, proceed with the exercise.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
90Example:
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
91
6/15/2007 Ravi Jayakaran / Participatory Evaluations
4
•Some members of the community have not got the techniques of SRI
And the improved Veterinary techniques
•Technique of SRI
•Technique of the Veterinarian
•Know How to take care of their own health
•Know how to increase their income
•Know about being Gender sensitive
•Have good management of their own families
DetailsTSTDetailsTST
Yet to be achievedAchieved already
Macro picture in the 76 villages where CHE works
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
92
Steps involved: Ask the Focus group to group the seeds together into 2 groups, one depicting the impact that has already taken place and the other showing the impact that is yet to take place. After there is consensus on this, the group is asked to give details of actual and specific impact in terms of new acquisitions, innovations and changes under impact achieved and areas of shortcomings in the second group.
Analysis of the information: In the example above, we find that around 60 % of expected impact has been achieved, while 40 % on the impact is still to be achieved. Details of these provide us information on what can be celebrated and what still needs focused attention for future improvement through greater or modified effort.Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
93
Code Number: PE- 04 Time required: 20-25 minutes
Name of the exercise: Impact-4: Community infrastructure provided.
Materials required: Flip charts, marker pens, seeds, masking tape.
Preparation: Read the Briefing notes on the TST, and after the focus group of Program staff responsible for this impact has been briefed about what is expected of them, proceed with the exercise.
Example:
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
94
6/15/2007 Ravi Jayakaran / Participatory Evaluations
5
•Community Bridge
•Road
•Big irrigation system
•Many more schools required
•Big sewage system
•Many more Churches required
•Electricity
•Hospital
•Clean water systems for larger populations
•Farms
•New school building
•Church constructed
•Village path developed
•Health centre constructed
•Sewage system developed
•Irrigation system developed
•Pond dug
•Small farm established
DetailsTSTDetailsTST
Yet to be achievedAchieved already
Macro picture in the 76 villages where CHE works
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
95
Steps involved: Ask the Focus group to group the seeds together into 2 groups, one depicting the impact that has already taken place and the other showing the impact that is yet to take place. After there is consensus on this, the group is asked to give details of actual and specific impact in terms of new acquisitions, innovations and changes under impact achieved and areas of shortcomings in the second group.
Analysis of the information: In the example above, we find that only around 20 % of expected impact related to community infrastructure needs has been achieved, while 80 % on the impact is still to be achieved. Details of these provide us information on what can be celebrated and what still needs focused attention for
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
96future improvement through greater or modified effort. However in CGA-CHE's case this activity is used more as a way to provide some work opportunity as relief, after diasasters. In this sense the objective has been met.Code Number: PE- 05 Time required: 20-25 minutes
Name of the exercise: Macro-profile of expenditure analysis in the area of work.
Materials required: Flip charts, marker pens, seeds, masking tape.
Preparation: Read the Briefing notes on the TST, and after the focus group of Program staff responsible for this impact has been briefed about what is expected of them, proceed with the exercise.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
97Example:
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
98
6/15/2007 Ravi Jayakaran / Participatory Evaluations
6
Other Other Expenses Expenses (ceremonies, (ceremonies, gasoline etc)gasoline etc)
Health Health carecare
SchoolSchoolFoodFood
Macro picture – CHE Project area (76 villages)
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
99
Steps involved: Ask the Focus group to group the seeds together into various groups, depicting the various things that an avarage family in the community has by way of expenditire. After there is consensus on the ratios and components, a diagram is made as shown above.
Analysis of the information: In the example above, we find that around 40% of the community's expenditure is on food, followed by 30% on health care and 20% on education. Other expenditure related to transport, ceremoies etcetra account for another 10%. As a thumb rule, a community that has a third of its expence on food is considered to be a poor community. This community also spends a lot on health
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
100care, hence anything done to reduce the cost of health care will also have a major impact on the community.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
101
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
102
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
103
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
104
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
10546. Briefing notes on the TST (Ten Seed Technique) developed by Dr. Ravi Jayakaran (QPI-GMS®: (Extract from The ‘Participatory Poverty Alleviation & Development ‘ book written By Dr. Ravi Jayakaran and published by World Vision International China, April 2003 & Mekong Institute, Khon Kaen University, Thailand: CD ROM 2005.
INTRODUCTION: The 10 seed technique is a modified PLA tool and was introduced after a lot of modification and experimentation as a tool that can be used to carry out the PLA-Participatory Learning and Action exercises. It is useful in gathering qualitative information on various issues, especially related to the perceptions of the
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
106community and the way people see themselves in relation to others. The technique is very flexible and therefore versatile enabling its use in combination with other techniques and also for collecting a wide range of information. The 10 seed technique enables probing deeply into different dimensions of an issue, for carrying out what is referred to as “opening up” the information. This essentially involves going deeper into an issue after starting at the absolute basic level. For example we find out about the health status of a community and then go deeper into it to find out reasons for difference, link it with their wealth status and go further into exploring the type of health care each group is able to access. This ‘opening up’ process can keep continuing as we find linkages to education level and attitudes.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
107
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
The Left and Right Brain…
•Input device (H/S)
•GIGO
•Trained
•Schooled
•Formatted
•Filing: Systematic
•Limited by the ‘schooling it gets’
•Input device (H/S/T/T/S)
•Visual
•Perceptive
•Analytical
•Versatile
•Filing: ”creative”
•Immense capability
•Access to sub-conscious mind
LEFT RIGHT
Ravi Jayakaran/QPI-GMS/ DFID-UNDP-MoEYS training/Feb 2006
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
108
Most of the traditional PLA exercises can be done using the 10 seed technique; hence this will be explained in considerable detail. Detailed information can be obtained from the web sites mentioned at the end of this chapter.
PHILOSOPHY BEHIND THE TECHNIQUE: This technique relies on using the Right brain function so that the full potential of the brain for perceptive analysis is utilized. The right brain is initiated into action by visuals such as pictures, and three-dimensional items. When we use seeds to depict aspects of information, the visual created by the seeds (strong contrast of colors
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
109between the seeds and background), helps the Right brain play a dominant role in the analysis. In the beginning when this technique was developed, it was designed to enable illiterate villagers to participate in the discussions related to analyzing their situation. The idea was to remove differences due to literacy and enable those without literacy to be able to participate equally with those who were literate. However when the technique was used in the field, it was discovered that there were additional benefits from being able to activate the right brain. The right brain is the more powerful part of the brain, because it is the creative side, more perceptive, more analytical and also has access to the information both in the left-brain and the subconscious mind. Thus today the technique is even used with those who are
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
110literate! For best results, facilitation must aim at getting the group to ‘move the seeds first’ then describe the section…rather than preparing a list first and then trying to slot the seeds in!!
POTENTIAL USES OF THE TECHNIQUE: As has been already mentioned, the 10 seed technique can be used for a variety of exercises for information collection. These are for example: Profile of street children, Vaccination levels, Patterns of distribution among a population (health care, HIV/AIDS, birth control practices, etc.), disease incidence, agriculture patterns, animal husbandry practices, sanitation practices, housing needs, MED profile in an
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
111area, water resources, CEDC, rapid damage assessment in Disasters, education levels and, many other issues that project staff need to collect information about. Besides these, several of the routine PLA exercises can also be carried out by this technique for example: Trends analysis, Seasonality diagram, Livelihood analysis, Expenditure analysis, problem analysis, and etcetera. Some of the new exercises which are used for Wholistic World View Analysis (WWVA), and Capacity–Vulnerability (C/V) analysis, Rapid Food Security status assessment (RFSA), Gender desegregation, HIV/AIDS macro zoom, and District/County level planning also use the ten seed technique. These additional exercises have been designed because of specific needs in program management and other community needs.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
112Given below is a brief description of how the ten seed technique can be used for each of these exercises:
Trends analysis: The group is asked to think of all the things that have changed in their community since the past. They are then asked to list the things that have changed. In front of each issue that has changed they are asked to prepare two columns, representing the situation – “then” and “now’. For each area of Change, they are asked to use 10 seeds and distribute them between the ‘then’ and ‘now’ columns. The trend of change then becomes obvious and allow for a lot of discussions.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
113
Seasonality Diagram: The group is asked to carry out a seasonality diagram as in the standard practice, but with the change that for each of the seasonal events such as rainfall, agriculture, disease incidence, festivals, Labor opportunity etcetera, they are asked to use only ten seeds for distribution. This enables us to identify the occurrences according to percentage intensity at different times of the year.
Livelihood analysis: The group is asked to imagine the 10 seeds represent the entire income of the whole village from all sources, throughout the year. (To make it easier they are asked to imagine it all to be converted into money as some of it is
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
114generated in cash and some in materials). They are then asked to divide the seeds into groups representing the sources of the income. Use of the ten seeds here enables us to find out what the main livelihood sources are.
Expenditure analysis: The group is asked to imagine that the 10 seeds represent the total expenditure of the village for the whole year. They are then asked to group the seeds into clusters to show what those various expenditure heads are on a yearly basis. Again this allows us to determine the percentage of expenditure on various items such as food, clothes, and medical treatment, etcetera.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
115Problem analysis: The group is asked to imagine that the 10 seeds represent all the problems that are faced by the community as a whole. They are then asked to group the seeds to represent what these problems are.
Disease incidence: The group is asked to imagine that the ten seeds represent all the diseases that occur in the village throughout the year. Here too the numbers of seeds in each group show us the percentage of a particular disease in a year.
Wholistic World View Analysis (WWVA): This exercise is done by combining the Livelihood analysis and the problem analysis information of the village with the
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
116information on uncertainties in the village. This is an exercise that involves the whole village and is usually done after carrying out a debriefing to the whole village of information gathered in their village.
Rapid Food Security status assessment (RFSA): This technique is carried out using the ten seeds and the group is asked to classify the village families into different groups according to their Food Security Status and according to the periods which they have struggles generating their livelihood.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
117
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
118Gender desegregation: This exercise is carried out to identify issues that are gender related and to determine the extent to which there is a link of an issue or a responsibility with Gender. To find this out, the issue is first determined. Let us take for example the issue is the decision regarding Family planning. We then ask the group what extent of the decision on family planning is determined by the men and by the women. They are asked to divide the 10 seeds accordingly.
HIV/AIDS macro zoom: This is a method used to determine rapidly issues related to HIV/AIDS risk in a community. It is particularly useful for determining the strategy for HIV/AIDS prevention in a city. The group to work with is composed of around 25
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
119to 30 people drawn from different walks of life who are familiar with the city. These people are usually from the media, the health care services, the entertainment business, the law enforcement department, and students from the University, the transportation services and the NGO sector that has been working with HIV/AIDS prevention in the city. The series of exercises that are part of the Macro zoom look at the HIV/AIDS risk frame for the city and in the course of a day are able to make a rapid study of the potential strategies to be followed. The details of the exercises and examples are currently being documented by the author of this book, and should be available in due course of time.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
120
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
121
ADVANTAGE OF THE TECHNIQUE: The technique is very simple to understand and learn and equally easy to practice. When used with village groups also, they very easily understand it. It has been tested in several countries and has worked equally effectively in all of them. This technique has also been tested and found successful with people in different age groups varying from very young children to old people, with gender segregated groups and mixed groups and with people of no literacy and those with Doctorates!
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
122The other advantages are that seeds are easily available everywhere, very non-threatening, and easy to move and move again. Once the moving around of seeds stops with unanimous agreement the information can be written on a sheet of paper. The technique is a very visual one, and because of this allows for the literate and illiterate to participate as equal partners and contribute meaningfully to the discussion. The specific number of seeds enables the group to make reasonable comparisons. It is also possible because of this to determine approximate percentages. The resultant visuals are easy to explain, understand and discuss around.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
123How the technique works: After initial rapport building with the group and explaining to them that the purpose of the exercises is to understand and learn from them about their perspective; we proceed with the technique. The group is given the ten seeds and asked to consider them to represent the entire population under study. They are then asked to move the seeds around into groups representing the aspect being analyzed. Once the groups of seeds have been formed, the participants are asked to describe them and give details on reasons for classifying them the way they have done. Further details are then sought on indicators that determined the segregation. Each group of seeds now has a very
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
124Distinct identity accorded by the participants, and discussions can now proceed around the “visual” created. Discussions around the “visual” soon become very intense and animated. After finalization the information is transferred on paper for sharing with the larger group. The following slide shows an example of this: After this is done, we proceed further to ask more details, looking for
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
125example at the types of birth control measures used. The group is asked to look at the ‘visual’ and pointing to the 8 seeds representing the population that regularly practices birth control measures, we ask for them to divide these further in terms of the types of measures practiced. This can be done in two ways, either just asking them to divide the eight seeds, or by again taking ten seeds and asking them to consider these seeds to represent those who practice family planning measures regularly.The result of doing this can be seen in the following slide, where the group divided the seeds further into 5 groups showing the pattern of distribution. Thus, by looking at this it is possible to identifying what percentage of the overall population
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
126approximately uses a particular type of birth control measure. For example in the case above we found that the community was under the perception that they were quite well protected against HIV/AIDS, because a fairly high percentage of them were using birth control measures. They had, in their thinking, equated ‘protection’ against pregnancy as being protection against HIV/AIDS. Discussions around the visual then could proceed to understand why this was so.
Depending on the purpose of the exercise we can proceed further. As in the case mentioned above the purpose was to find out the condom usage prevalence rate, because another exercise in the same community had shown a fairly high level of
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
127promiscuous behavior among them. The exercise thus enabled us to see that only 30 % of the birth control measures in use were condoms, which also did not provide them 100% protection against the spread of HIV/AIDS.
Thus the exercise can also be the basis of discussions for modification of behavior when the community ‘discovers’ how much at risk it is. These discussions can also lead into understanding appropriate interventions in the community for modified or changed behavior. Other exercises can be linked to this to find out how the
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
128community gets its information on family planning and thus we can identify the best and most effective strategy for intervening in the community.
GROUP SIZE & PARTICIPATION: The ideal size of participants group is around 8-10 persons. Some would swear that the ideal size is 7-9 persons, but there is need to be flexible about this because these optimum sizes may not always be possible in the community, with the group sometimes being smaller or larger. In larger groups too the actual number of active participants may only be 8-10.If the number of active participants increases beyond that, and then it might be important to split the group further. Giving everyone an equal chance to share views can enable active
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
129participation. Sometimes it may be necessary to “filter out” the over dominating ones in the group, so that others can participate. Getting someone in the group to ‘diplomatically’ take the person away from the group for a KII-Key Informant interview does this. While this has the benefit that the group left behind becomes active in participation, good information can also be obtained from the person filtered out. Prime candidates for this type of filtering are usually Schoolteachers, Village chiefs, Businessmen, moneylenders and “educated” people in illiterate communities.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
130
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
131FACILITATING THE EXERCISES: Before going to the field to carry out the exercises, it is good to have the group prepare itself to work together. Besides establishing a code of conduct to respect and value one another, group members must also decide roles among themselves beforehand to determine who will be the ‘ Interviewer’, ‘the recorder’ and if the situation calls for it- the ‘Filter’!! As with other PRA/PLA exercises, facilitation involves ensuring that there is no dominance, and that everyone gets an equal opportunity to participate and that there is a balance between being open to new and divergent views as well as being focused enough to lead the discussions in the direction of the information being sought. Facilitation also involves determining when a dominant person has become the ‘defacto-
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
132spokesperson’ of the group and needs to be diplomatically ‘filtered’ out!! The facilitator (Interviewer) should also keep a balance of time required for the exercise so that it is neither hurried nor drags on too long!
KEY SUCCESS FACTORS: The correct attitude, a balanced group, adequate time and the ability to keep the group working with a right brain orientation are some of the Key success factors in getting good quality information. It is almost mandatory that the ‘outsiders’ carrying out the 10 seed technique exercises in the community have the correct attitude. This attitude is one of listening with a positive attitude. This essentially means being open to new perspectives and views without pre-
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
133assumptions which are merely seeking for an ‘affirmation’ from the community. A balanced group of outsiders with varied experience and one that does not those among it who are overly dominant, nor seek to dominate the community, is also important. Such a group will be able to enable the community to participate well and spontaneously. Adequate time should be available so that there is no need to rush. The exercise can then be coordinated and facilitated well, and focus on good discussions with openness to new views and perspectives.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
134It is also very important to keep the group working with a right brain orientation. This essentially means that at the beginning stages the seeds should be the main things on the paper with only symbols or diagrams drawn to mark the issues identified.
ASKING GOOD QUESTIONS: After establishing good rapport with the community group the facilitators of the exercise should explain the purpose of the exercise and the subject that they propose to explore with them. A foundational principle to remember in asking questions is to remember to ask questions to learn and understand, not to ‘affirm’ pre-assumptions. Questions asked should therefore be ‘Open ended’, and the 5W+1H (Who? What? Where? When? Why? + How?)
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
135Principle is a good one to adapt. The general principle is to start with the simplified foundational information first and then get deeper and deeper into the issue. Going deeper into issues to ‘Open up’ the information is a little more complex but comes easily with practice. Here too the best results are obtained by continuing to be as “visual” as possible and getting participants to move the seeds first and then describe the categories.
INTERVIEWING THE ‘VISUAL’ Once the ‘visual’ of the seeds placed in the different groups has been created it is reviewed with the participants to make sure everyone understands the placement of the seeds. After this various aspects of this are
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
136discussed…. this is done by “interviewing” the visual. This is a very non-threatening method, as we don’t need to make embarrassing eye contact with the participants especially when collecting sensitive information. When they feel comfortable and they are ready for it, they do however make eye contact, but this is on the basis of an established relationship. At the end of the session a participant from the community is invited to give a summary of the observations and findings. The document can be copied and the original left behind for use by the village in future.
PRECAUTIONS AND CARE: There are some precautions and care that one must take while using the 10 seed technique. The first and most important one in this is
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
137that the facilitator must have the correct attitude. This is Mandatory. Second, the purpose of the exercise must be explained to the community clearly right at the beginning.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
138
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
139Since the technique uses right brain function it has the potential to bring Hopes & Dreams to the surface and therefore something must be done to address them after the exercise. Similarly when seeking sensitive information, it may throw light on an exploitatory situation in the village that must be dealt with urgently. This too must be dealt with soon, or it will result in frustration and the oppression may continue unchecked. It is also essential to ensure equal participation and opportunity to speak for all participants without dominators ‘taking over complete’ control. Chapter six outlines ways in which to deal with such situations. Taking these precautions will enable the exercise to be done well and successfully.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
140POTENTIAL PROBLEMS: Dominant people in the community could be one potential big problem that will hinder good participation from the community. To prevent this, “Filter” dominators for KII (Key informant Interview) so that others get a chance to participate equally. As mentioned earlier, prime candidates for filtering are usually: Village leaders, moneylenders, Landlords, old school teachers, or other authorities. This, however, must be done sensitively so that the ‘filtered’ person still feels important. Besides, this is often a blessing in disguise, because the filtered person does in fact give some good information.Other problems can be avoided by being sensitive to the presence of people with vested interests in the group and ensuring that information is not biased in favor of their interests. Gender biased information
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
141should be countered by trying to get some gender balance. If the context is one in which women will hesitate to speak in the presence of men, and then divide the participants into 2 groups. Choose the timings for the exercise carefully so that it doesn’t disrupt the normal life of the village badly.
GOOD FACILITATION: This essentially involves being able to overcome some of the potential problems that one might face as mentioned earlier. The 10 seed technique is very easy to learn, but the correct attitude is hard to develop (hence we need to work on this if we want to be good facilitators). Developing openness to new perspectives and ideas and views is essential. One has to seek to listen and learn
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
142(not seek to confirm pre-determined ideas). Success is guaranteed when one develops a ‘listening attitude”. A good facilitator is sensitive to the local context and culture and ensures equal participation and counters dominance by individuals trying to become spokespersons for the group. If handled correctly, the exercises will generate animated discussions among the participants.
Let facilitation become a way of life for you…. not only at work, but also in your personal life!!
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
143‘OPENING UP’ THE INFORMATION:
This is done for issues that have ramifications with several factors for example the Food Security Status of a village in relation to migration patterns. The Food security status of the village is understood and then for each food security status level further details are found out for example level of children’s education, Health status, migration patterns, use of different programs etcetera. Carrying out this exercise results in the creation of a complex and complicated diagram. This diagram shows the ramifications and the underlying issues involved that cause the particular situation. Since the complex diagram emerged from the simple it is
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
144easy to understand, describe and discuss for all participants, even the villagers with low or no literacy.USING THE INFORMATION: Information generated by using the 10 seed technique must be used as soon as possible in planning and designing. However, oftentimes what really happens is that those who collect it are unable to analyze it and thus don’t know how to use it. When using technique for analyzing specific issues such as say the level of vaccination, the resulting information can be used to generate a Plan of Action for getting the unvaccinated children vaccinated. The participants of the exercise can be asked how to ensure that this happens for each level identified.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
145When the entire series of exercises is conducted, it must result in the formulation of a Village Resource Development Plan (VRDP) .The Problem analysis and Livelihood analysis exercise of the village in combination with the uncertainty profile of the village can be used to prepare for the Wholistic World View Analysis (WWVA), which in turn can help is generating a C/V (Capacities & Vulnerabilities) analysis of the village for Development planning. Sometimes the exercise is done to identify the causes of a specific problem. Thus the information generated can immediately be used by addressing the identified causes. On other occasions the exercise is done to find out why a particular program is not succeeding…. the approach here would then be to make modifications as suggested by the community.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
146
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
147
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
148The progress of the program against planned objectives can be done in the form of an evaluation .For example for a particular objective if the planned impact was ‘x’ and we represent this with ten seeds. We then ask the community how many seeds would represent the actual impact, and let us say they put 7 seeds to show this; we then ask them to list these. Similarly, we can then proceed to ask them what modification and emphasis (3 seeds) needs to be given in the project for that component in future to bring it up to speed. This could even be done at each output level for the project.
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
149CONCLUSIONS: The range of uses that the 10 seed technique can be put to have been clearly detailed in this book and the one specifically related to the TST. Their uses range from participatory information gathering to situation analysis, planning and evaluation. Extended applications have also been explained for example on how it can be used to deal with specific issues such as HIV/AIDS risk, Food Security Status, understanding the needs of children etcetera.
Web site references:
http://www.fao.org/Participation/ft_show.jsp?ID=1981
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
150http://www.childsurvival.com/connections/start.cfm#spring2002http://www.fupin.org.cn/ngohy/ngo-z/ngo-z-show-16.htmhttp://www.mrcmekong.org/info_resources/infores002b004.htmhttp://www.hivmekong.net/Downloads/Toolkits/Toolkit_Final.pdf
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
151
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
152
Glossary of some terms used in this book:
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
153
Additional reading on the subject:
No Title Author
01 Rural Development – Putting the last first(Longman, Harlow)
Dr Robert Chambers1983
02 Participatory Workshops(Earth scan)
Dr Robert Chambers2002
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Filtering The process by which a dominant person is diplomatically removed from a group
FGD Focused Group Discussion
HIV/AIDS Human Immuno deficiency Virus/ Acquired Immuno Deficiency Syndrome
KII Key Informant Interview: Where the source of information is a single key informant
Optional ignorance
Choosing to ignore certain aspects of information that are not relevant to analysis or planning
Optimal ignorance
The maximum that one can safely ignore in investigating
OSPE Out of school Peer Educators
OSP Out of School Peers
PRA Participatory Rural Appraisal
PLA Participatory Learning and Action
PPE Primary Peer Educators
PP Primary Peers
PLWHA People Living With HIV/AIDS
PPA&E Participatory Process Assessment & Evaluation: a technique by which we analyze where the development process or intervention is at in the ‘continuum’. This ‘in-course’ assessment enables us to see how the project is progressing.
SSI Semi Structured Interview: This type of interview involves leading the interview in a particular direction with questions that first start off on general issues and then slowly focus on specific issues within the general frame.
SPE Secondary Peer Educators
SP Secondary Peers
TST Ten Seed Technique
Triangulation The process by which other perspectives are also sought to ensure that the information collected is not biased or one sided.
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
154
03 Whose reality counts? Putting the first last(Intermediate Technology publication)
Dr Robert Chambers1997
04 PRA/PLA Notes series, (especially RRA notes 13), London, 1991
IIED: International Institute of Environment & Development
05 PLA User guide & Manual(World Vision India publication)
Dr Ravi Jayakaran1996
06 PLA – a five minute guide(WVI / AusAid HIV/Aids toolkit)
Dr Ravi Jayakaran1996, 2001
07 Use of PLA for collecting sensitive information(AIDSCAP – India)
Dr Ravi Jayakaran1998
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
155
08 TST- The Ten Seed Technique(WVI China/ WVA downloadable) English/Chinese
Dr Ravi Jayakaran2002
09 Participatory Poverty Alleviation & Development(WVI China/ Mekong Institute, KKU, Thailand)English /Chinese
Dr Ravi Jayakaran2003/2005 CD ROM
10 Facilitating Small Groups(WVI China/ Mekong Institute, KKU, Thailand)English / Chinese
Dr Ravi Jayakaran2003/2005 CD ROM
11 A Trainer’s guide to Participatory Learning and action(IIED series)
Dr Jules Pretty Et all1995
12 Stepping stones: A training package on HIV/AIDS, Dr Alice Welbourne
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
156gender issues, communication and learning skills(Action Aid, London)
1996
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
157
A short note on writing case studies:
Case studies are the best way of getting deeper into an issue to understand the situation as well as the circumstances. This is particularly useful in determining the impact that has taken place in the life of an individual. As with any good interview, use the 5W and 1H principle to determine the questions that you will ask to find out
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
158as much about the person and his/her situation and circumstances. Since there is likely to be a lot of sensitive information that will be generated from this type of case study, take the following precautions:
Don’t use the person’s real name if issue is sensitive (change it for a fictitious name)
When taking photographs ensure that the person’s identity cannot be determined from looking at it
In the case study try to determine specific instances and decisions that resulted in action on the part of the individual, to understand the mental profile and mental process that precipitated the action
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
159 The case study also throws up some excellent perspectives on the ‘realities
on the ground’
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
160
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
161
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
162
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
163
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
164
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
165
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual
Participatory Evaluation: A user guide and ManualSpecially prepared fCambodia Global Action and Samaritan's Purse International
166
Dr. Ravi Jayakaran/QPI-GMS® /Participatory Evaluation: A user guide and Manual